Systems and methods for controlling autonomous vehicles that provide a vehicle service to users

ABSTRACT

Systems and methods for controlling autonomous vehicles are provided. In one example embodiment, a computer implemented method includes obtaining data indicative of a location associated with a user to which an autonomous vehicle is to travel. The autonomous vehicle is to travel along a first vehicle route that leads to the location. The method includes obtaining traffic data associated with a geographic area that includes the location. The method includes determining an estimated traffic impact of the autonomous vehicle on the geographic area based at least in part on the traffic data. The method includes determining vehicle action(s) based at least in part on the estimated traffic impact and causing the autonomous vehicle to perform the vehicle action(s) that include at least one of stopping the autonomous vehicle at least partially in a travel way within a vicinity of the location or travelling along a second vehicle route.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is based on and claims priority to U.S.Provisional Application 62/510,515 having a filing date of May 24, 2017,which is incorporated by reference herein.

FIELD

The present disclosure relates generally to controlling the travelholding pattern of an autonomous vehicle that provides a vehicle serviceto a user.

BACKGROUND

An autonomous vehicle is a vehicle that is capable of sensing itsenvironment and navigating without human input. In particular, anautonomous vehicle can observe its surrounding environment using avariety of sensors and can attempt to comprehend the environment byperforming various processing techniques on data collected by thesensors. Given knowledge of its surrounding environment, the autonomousvehicle can identify an appropriate motion path through such surroundingenvironment.

SUMMARY

Aspects and advantages of embodiments of the present disclosure will beset forth in part in the following description, or may be learned fromthe description, or may be learned through practice of the embodiments.

One example aspect of the present disclosure is directed to acomputer-implemented method of controlling autonomous vehicles. Themethod includes obtaining, by a computing system that includes one ormore computing devices, data indicative of a location associated with auser to which an autonomous vehicle is to travel. The autonomous vehicleis to travel along a first vehicle route that leads to the locationassociated with the user. The method includes obtaining, by thecomputing system, traffic data associated with a geographic area thatincludes the location associated with the user. The method includesdetermining, by the computing system, an estimated traffic impact of theautonomous vehicle on the geographic area based at least in part on thetraffic data. The method includes determining, by the computing system,one or more vehicle actions based at least in part on the estimatedtraffic impact. The method includes causing, by the computing system,the autonomous vehicle to perform the one or more vehicle actions. Theone or more vehicle actions include at least one of stopping theautonomous vehicle at least partially in a travel way within a vicinityof the location associated with the user or travelling along a secondvehicle route.

Another example aspect of the present disclosure is directed to acomputing system for controlling autonomous vehicles. The computingsystem includes one or more processors and one or more memory devices.The one or more memory devices store instructions that when executed bythe one or more processors cause the computing system to performoperations. The operations include obtaining data indicative of alocation associated with a user. The user is associated with a requestfor a vehicle service provided by an autonomous vehicle. The autonomousvehicle is to travel along a first vehicle route to arrive within avicinity of the location associated with the user. The operationsinclude obtaining traffic data associated with a geographic area thatincludes the location associated with the user. The operations includeobtaining location data associated with a user device associated withthe user. The operations include determining at least one of anestimated traffic impact of the autonomous vehicle on the geographicarea based at least in part on the traffic data or an estimated time ofuser arrival based at least in part on the location data associated withthe user device. The operations include determining one or more vehicleactions based at least in part on at least one of the estimated trafficimpact or the estimated time of user arrival. The operations includecausing the autonomous vehicle to perform the one or more vehicleactions. The one or more vehicle actions include at least one ofstopping the autonomous vehicle at least partially in a travel waywithin a vicinity of the location associated with the user or travellingalong a second vehicle route.

Yet another example aspect of the present disclosure is directed to anautonomous vehicle includes one or more sensors, a communication system,one or more processors, and one or more memory devices. The one or morememory devices store instructions that when executed by the one or moreprocessors cause the autonomous vehicle to perform operations. Theoperations include obtaining data indicative of a location associatedwith a user. The user is associated with a request for a vehicle serviceprovided by the autonomous vehicle. The operations include controllingthe autonomous vehicle to travel along a first vehicle route to arrivewithin a vicinity of the location associated with the user. Theoperations include obtaining traffic data associated with a geographicarea that includes the location associated with the user based at leastin part on sensor data obtained via the one or more sensors. The trafficdata is indicative of a level of traffic within a surroundingenvironment of the autonomous vehicle. The operations include obtaining,via the communication system, location data associated with a userdevice associated with the user. The location data associated with theuser device is indicative of one or more locations of the user deviceassociated with the user at one or more times. The operations includedetermining an estimated traffic impact of the autonomous vehicle on thegeographic area based at least in part on the traffic data. Theoperations include determining an estimated time of user arrival basedat least in part on the location data associated with the user device.The operations include determining one or more vehicle actions based atleast in part on at least one of the estimated traffic impact or theestimated time of user arrival. The operations include causing theautonomous vehicle to perform the one or more vehicle actions. The oneor more vehicle actions include at least one of stopping the autonomousvehicle within the vicinity of the location associated with the user ortravelling along a second vehicle route.

Other example aspects of the present disclosure are directed to systems,methods, vehicles, apparatuses, tangible, non-transitorycomputer-readable media, and memory devices for controlling autonomousvehicles.

These and other features, aspects and advantages of various embodimentswill become better understood with reference to the followingdescription and appended claims. The accompanying drawings, which areincorporated in and constitute a part of this specification, illustrateembodiments of the present disclosure and, together with thedescription, serve to explain the related principles.

BRIEF DESCRIPTION OF THE DRAWINGS

Detailed discussion of embodiments directed to one of ordinary skill inthe art are set forth in the specification, which makes reference to theappended figures, in which:

FIG. 1 depicts an example system overview according to exampleembodiments of the present disclosure;

FIG. 2 depicts an example geographic area that includes a locationassociated with a user according to example embodiments of the presentdisclosure;

FIG. 3 depicts example information associated with an acceptable walkingdistance according to example embodiments of the present disclosure;

FIG. 4 depicts an example display device with an example communicationaccording to example embodiments of the present disclosure;

FIG. 5 depicts example information associated with an estimated trafficimpact according to example embodiments of the present disclosure;

FIG. 6 depicts an example travel way according to example embodiments ofthe present disclosure;

FIG. 7 depicts a flow diagram of an example method of determining anestimated time of user arrival according to example embodiments of thepresent disclosure;

FIG. 8A depicts an example portion of a communications system accordingto example embodiments of the present disclosure;

FIG. 8B depicts an example portion of a communications system accordingto example embodiments of the present disclosure;

FIG. 8C depicts an example diagram of obtaining location data accordingto example embodiments of the present disclosure;

FIG. 9 depicts example information associated with a second vehicleroute according to example embodiments of the present disclosure;

FIG. 10 depicts an example display device with an example communicationaccording to example embodiments of the present disclosure;

FIG. 11 depicts example information associated with a vehicle servicecancellation threshold according to example embodiments of the presentdisclosure;

FIG. 12 depicts a flow diagram of an example method of controllingautonomous vehicles according to example embodiments of the presentdisclosure;

FIGS. 13A-B depict a flow diagram of an example method of controllingautonomous vehicles according to example embodiments of the presentdisclosure; and

FIG. 14 depicts example system components according to exampleembodiments of the present disclosure.

DETAILED DESCRIPTION

Reference now will be made in detail to embodiments, one or moreexample(s) of which are illustrated in the drawings. Each example isprovided by way of explanation of the embodiments, not limitation of thepresent disclosure. In fact, it will be apparent to those skilled in theart that various modifications and variations can be made to theembodiments without departing from the scope or spirit of the presentdisclosure. For instance, features illustrated or described as part ofone embodiment can be used with another embodiment to yield a stillfurther embodiment. Thus, it is intended that aspects of the presentdisclosure cover such modifications and variations.

Example aspects of the present disclosure are directed to improving thetravel patterns of an autonomous vehicle to account for potentialtraffic impacts, while waiting to provide a vehicle service to a user.For instance, an entity (e.g., service provider) can use a fleet ofvehicles to provide a vehicle service (e.g., transportation service,delivery service, courier service, etc.) to a plurality of users. Thefleet can include, for example, autonomous vehicles that can drive,navigate, operate, etc. with minimal and/or no interaction from a humandriver. For example, an autonomous vehicle can receive data indicativeof a location associated with a user that has requested a vehicleservice, such as a transportation service. The autonomous vehicle canautonomously navigate along a route towards the location associated withthe user. When within a vicinity of the location, the autonomous vehiclecan attempt to identify a parking spot that is out of a travel way(e.g., out of a traffic lane). Such parking spots may not, however, beavailable to the autonomous vehicle. In such a case, the autonomousvehicle can determine whether the vehicle should stop in a travel laneto pick up the user and/or whether the vehicle should enter a holdingpattern whereby the vehicle continues to travel (e.g., around the block)to return to the user's location. For example, as further describedherein, the autonomous vehicle can estimate the impact the autonomousvehicle may have on traffic if the autonomous vehicle were to stop in atravel way to wait for the user, given, for example, the user'sproximity to the vehicle. In the event that the impact would be low, theautonomous vehicle can stop in the travel way and alert the user of thevehicle's location. In the event that the impact would be high, orbecomes high after the vehicle has stopped, the autonomous vehicle canbe re-routed (e.g., around the block) so that the vehicle can againattempt to pick up the user. In this way, the system and methods of thepresent disclosure can improve the situational awareness of anautonomous vehicle that is waiting for a user of a vehicle service(e.g., while attempting to pick up a user for transport).

More particularly, an entity (e.g., service provider, owner, manager)can use one or more vehicles (e.g., ground-based vehicles) to provide avehicle service such as a transportation service (e.g., rideshareservice), a courier service, a delivery service, etc. The vehicle(s) canbe autonomous vehicles that include various systems and devicesconfigured to control the operation of the vehicle. For example, anautonomous vehicle can include an onboard vehicle computing system foroperating the vehicle (e.g., located on or within the autonomousvehicle). The vehicle computing system can receive sensor data fromsensor(s) onboard the vehicle (e.g., cameras, LIDAR, RADAR), attempt tocomprehend the vehicle's surrounding environment by performing variousprocessing techniques on the sensor data, and generate an appropriatemotion plan through the vehicle's surrounding environment. Moreover, theautonomous vehicle can be configured to communicate with one or morecomputing devices that are remote from the vehicle. For example, theautonomous vehicle can communicate with an operations computing systemthat can be associated with the entity. The operations computing systemcan help the entity monitor, communicate with, manage, etc. the fleet ofvehicles.

An autonomous vehicle can be configured to operate in a plurality ofoperating modes. For example, an autonomous vehicle can be configured tooperate in a fully autonomous (e.g., self-driving) operating mode inwhich the autonomous vehicle can drive and navigate with no interactionfrom a human driver present in the vehicle. In some implementations, ahuman driver may not be present in the autonomous vehicle. Theautonomous vehicle can also be configured to operate in an approach modein which autonomous vehicle performs various functions as it approachesa location associated with a user, such as searching its surroundingenvironment for a parking location. The approach operating mode can beutilized, for example, when the autonomous vehicle is approaching a userthat has requested a vehicle service, as further described herein.

A user can make a request for a vehicle service provided by theautonomous vehicle. For instance, a user can provide (e.g., via a userdevice) a request to the operations computing system of an entity (e.g.,service provider, manager, owner) that is associated with the autonomousvehicle. The request can indicate the type of vehicle service that theuser desires (e.g., a transportation service, a delivery service, acourier service, etc.), a location associated with the user (e.g., acurrent location of the user, a different location, etc.), an identifier(e.g., phone number, Bluetooth, WiFi, Cellular, other data that can beused to contact the user, etc.) associated with the user device thatprovided the request, and/or other information.

The operations computing system can process the request and select anautonomous vehicle to provide the requested vehicle service to the user.The operations computing system can provide, to the autonomous vehicle,data indicative of a location to which the autonomous vehicle is totravel. The location can be associated with the user that requested thevehicle service. For example, the location can be the current locationof the user and/or a different location, such as for example a locationat which the user would like to be picked up by the autonomous vehicle,provide an item to the autonomous vehicle, retrieve an item from theautonomous vehicle, etc. The location can be expressed as a coordinate(e.g., GPS coordinate, latitude-longitude coordinate pair), an address,a place name, and/or other geographic reference that can be used toidentify the location. The location associated with the user can berepresented, for example, as a pin on a map user interface.

The autonomous vehicle can obtain, from the operations computing system,the data indicative of the location associated with the user. Theautonomous vehicle can also obtain a first vehicle route that leads tothe location associated with the user. The first vehicle route can be,for example, a route from the current location of the vehicle to thelocation associated with the user. In some implementations, theoperations computing system can provide the first vehicle route to theautonomous vehicle. Additionally, or alternatively, the onboard vehiclecomputing system of the autonomous vehicle can determine the firstvehicle route.

The autonomous vehicle can travel in accordance with the first vehicleroute to arrive within a vicinity of the location associated with theuser. The vicinity can be defined by a distance (e.g., a radialdistance) from the location associated with the user. The distance canbe indicative of an acceptable walking distance from the locationassociated with the user. For example, an acceptable walking distancecan be distance that a user would be willing walk to arrive at avehicle. The acceptable walking distance can be determined based on avariety of information. Such information can include, for example,specific user preferences stored in a profile associated with the user,weather information (e.g., gathered via sensors onboard the vehicle,provided by a third party source, etc.), traffic conditions (e.g.,current, historic, future predicted traffic conditions), historicvehicle services data (e.g., historic pickup data for previoustransportation services), and/or other types of information. Theautonomous vehicle can provide a communication to the user indicatingthat the autonomous vehicle is within the vicinity of the locationassociated with the user. For instance, such a communication can be inthe form of a textual message stating “your ride is arriving pleaseprepare to board”.

Once the autonomous vehicle is within the vicinity of the locationassociated with the user, the autonomous vehicle can begin to search fora parking location. For example, the autonomous vehicle can enter intothe approach operating mode when the vehicle is within the vicinity ofthe location associated with the user. While it is within the vicinity,the autonomous vehicle can search for a parking location before itreaches the location associated with the user (e.g., before the GPS pincoordinate on a map) and/or after it passes the location associated withthe user, but is still within the vicinity of the user (e.g., withinacceptable walking distance for the user).

To identify a parking spot, the autonomous vehicle can obtain sensordata associated with one or more objects that are proximate to thevehicle (e.g., within a field of view of one or more of the vehicle'sonboard sensor(s)). The sensor data can include image data, radar data,LIDAR data, and/or other data acquired by the vehicle's sensor(s). Theobject(s) can include, for example, pedestrians, vehicles, bicycles,and/or other objects. The sensor data can be indicative of locationsassociated with the object(s) within the surrounding environment of thevehicle at one or more times. The autonomous vehicle can process thesensor data to determine if there are any available parking locationsthat are not currently occupied by the objects (e.g., other vehicles)within the vehicle's surrounding environment. In some implementations,the autonomous vehicle can utilize map data to determine if there areany designated parking locations (e.g., parking lots, pullover lanes,etc.) within the vicinity of the location associated with the user.

In the event the autonomous vehicle is able to identify a parkinglocation that is out of a travel way and within the vicinity of thelocation associated with the user (e.g., out of a traffic lane), theautonomous vehicle can position itself into that parking locationaccordingly. The autonomous vehicle can send a communication to a userdevice associated with the user. The communication can indicate that thevehicle has arrived as well as the location of the autonomous vehicle.In some implementations, the user device can display a map userinterface that includes a user route. The user route can be a routealong which a user can travel to arrive at the autonomous vehicle. Inthe event that the autonomous vehicle is unable to identify a parkinglocation that is out of the travel way (and within the vicinity of theuser), the autonomous vehicle can decide whether or not to stop at leastpartially in a travel way to wait for the user.

To help decide whether to stop within a travel way, the autonomousvehicle can obtain traffic data associated with a geographic area thatincludes the location associated with the user. The traffic data caninclude various types of data such as historic traffic data, predictedtraffic data, and/or current traffic data associated with the geographicarea (e.g., within the vicinity of the location of the user, a widerarea, etc.). The traffic data can be obtained from a variety of sourcessuch as other autonomous vehicles (e.g., within the vehicle fleet), theoperations computing system, third party sources (e.g., regional trafficmanagement entities, etc.), as well as the autonomous vehicle itself.

By way of example, the autonomous vehicle can obtain the current trafficdata associated with the geographic area that includes the locationassociated with the user. To do so, the autonomous vehicle can obtainsensor data (e.g., via its onboard sensors) associated with thesurrounding environment of the autonomous vehicle that is within thevicinity of the location associated with the user, as described herein.The sensor data can be indicative of one or more objects within thesurrounding environment of the autonomous vehicle. The autonomousvehicle can process the sensor data to classify which of the object(s)would be impacted (e.g., caused to stop) by the autonomous vehiclestopping in a travel way. For example, the autonomous vehicle canclassify the vehicles that are behind the autonomous vehicle (in thesame travel lane) as objects that would be impacted in the event theautonomous vehicle were to stop at least partially in the travel way.The autonomous vehicle can also identify object(s) that would not beaffected by the vehicle stopping in the travel way. For example, theautonomous vehicle can determine that object(s) that have a path totravel around the autonomous vehicle (e.g., bicycles, vehicles inadjacent lanes, vehicles with clear paths to change lanes, etc.), maynot be impacted and/or may be impacted to an insignificant degree. Aftersuch classification, the autonomous vehicle can determine a level oftraffic associated with the geographic area (e.g., within the vicinityof the user's location) based at least in part on the sensor data. Forexample, the level of traffic can be based at least in part on thenumber of object(s) within the surrounding environment of the autonomousvehicle that would be impacted by the autonomous vehicle stopping atleast partially in the travel way, while filtering out those object(s)that would not be impacted. Similar such information could be acquiredvia one or more other vehicles in the associated vehicle fleet.

The autonomous vehicle can determine an estimated traffic impact of theautonomous vehicle on the geographic area based at least in part on thetraffic data. The estimated traffic impact can be indicative of anestimated impact of the autonomous vehicle on one or more objects withina surrounding environment of the autonomous vehicle in the event thatthe autonomous vehicle were to stop at least partially in the travel way(e.g., in the vicinity of the location associated with the user). Forexample, in some implementations, the autonomous vehicle can compare thelevel of traffic (e.g., based on the sensor data) to a trafficconstraint to determine whether the estimated traffic impact would behigh or low. The traffic constraint can be implemented in a variety offorms. For example, the traffic constraint can include a trafficthreshold that is indicative of an acceptable level of traffic (e.g., anacceptable number of objects) that would be impacted by the autonomousvehicle stopping at least partially in the travel way. A traffic levelthat exceeds the traffic threshold would be considered a high impact ontraffic. In some implementations, the traffic constraint can beimplemented as cost data (e.g., one or more cost function(s)). Forexample, the autonomous vehicle's onboard vehicle computing system caninclude cost data that reflects the cost(s) of stopping vehicle motion,the cost(s) of causing traffic build-up, the cost(s) of illegallystopping in a travel way, etc.

The traffic constraint can be based on a variety of information. In someimplementations, the traffic constraint can be based at least in part onhistoric traffic data that indicates the level of traffic previouslyoccurring in that geographic area. For example, if the geographic areanormally experiences a high level of traffic build-up, a correspondingtraffic threshold can be higher (and/or the cost of stopping can belower). Additionally, or alternatively, the traffic constraint can bebased at least in part on real-time traffic data (e.g., from othervehicles in the fleet, from the autonomous vehicle, other sources). Forexample, in the event that there is already a traffic jam in thevicinity of the location of the user, the traffic threshold could behigher (and/or the cost of stopping could be lower). In someimplementations, the traffic constraint can be based at least in part onthe typical travel expectations of individuals in the geographic area.For example, individuals that are located in City A may be more patientwhen waiting in traffic than those in City B. Thus, a traffic thresholdmay be higher in City A than in City B (and/or the cost of stopping maybe lower in City A than in City B). In some implementations, the trafficconstraint can be based at least in part on map data. For example, inthe event that the autonomous vehicle is traveling on a wide travel wayin which impacted vehicles could eventually travel around the autonomousvehicle, the traffic threshold could be higher (and/or the cost ofstopping could be lower).

In some implementations, the traffic constraint can be determined atleast in part from a model, such as a machine-learned model. Forexample, the machine-learned model can be or can otherwise include oneor more various model(s) such as, for example, models using boostedrandom forest techniques, neural networks (e.g., deep neural networks),or other multi-layer non-linear models. Neural networks can includerecurrent neural networks (e.g., long short-term memory recurrent neuralnetworks), feed-forward neural networks, and/or other forms of neuralnetworks. For instance, supervised training techniques can be performedto train the model (e.g., using previous driving logs) to determine atraffic constraint based at least in part on input data. The input datacan include, for example, traffic data as described herein, map data,data from a traffic management entity, driving characteristics ofindividuals in an associated geographic area, complaints received fromoperators of vehicles that were caused to stop by autonomous vehicles,etc. The machine-learned model can provide, as an output, dataindicative of a recommended traffic constraint. In some implementations,the recommended traffic constraint can be specific to the geographicarea.

The autonomous vehicle can also, or alternatively, determine anestimated time of user arrival in order to help determine whether or notto stop at least partially in the travel way. For instance, theautonomous vehicle can obtain location data associated with a userdevice associated with a user. The location data can be indicative ofthe position of the user device associated with the user. The autonomousvehicle can use one or more identifier(s) of the user device (e.g.,provided by the operations computing system) to scan for and/orcommunicate with the user device when the vehicle is within the vicinityof the user. For example, the autonomous vehicle can utilize multipleinput/multiple output communication, Bluetooth low energy protocol, RFsignaling, and/or other communication technologies to obtain thelocation data. The autonomous vehicle can determine the estimated timeof user arrival based at least in part on the location data associatedwith the user device. The estimated time of user arrival can beindicative of, for example, a time at which the user is estimated tocomplete boarding of the autonomous vehicle (e.g., for a transportationservice). The estimated time of user arrival can be expressed as a timeduration (e.g., user estimated to arrive in 1 minute) and/or a point intime (e.g., user estimated to arrive at 10:31 am (PT)). In this way, theautonomous vehicle can determine an amount of stopping time that theobject(s) within its surroundings would be impacted as the autonomousvehicle waits for the user's arrival (e.g., how long other vehicleswould be caused to stop while waiting for the user).

By way of example, the operations computing system can provide theautonomous vehicle with one or more identifier(s) of a user deviceassociated with the user. The autonomous vehicle can use theidentifier(s) (e.g., Bluetooth, WiFi, Cellular, other identifier) todetermine the location of the user. For example, when the autonomousvehicle is within a vicinity of the location associated with the user,the autonomous vehicle can scan for the user device based at least inpart on the identifier(s). Once the user device is found, the autonomousvehicle can track changes in the signal strength (e.g., radio signalstrength identifier) to determine the approximate heading of the user aswell as the approximate distance between the user and autonomous vehicle(e.g., without authenticated connection). In some implementations, theautonomous vehicle can determine differences in Bluetooth Low Energybeacon radio signal strength identifiers over time and/or inertialmeasurement unit changes, which can indicate distance and direction ofthe autonomous vehicle from the user device (e.g., mobile phoneassociated with the user). The autonomous vehicle can calculate theestimated time of user arrival based at least on the heading of the userand the approximate distance between the user and the vehicle (and/orthe estimated velocity of the user), as further described herein. Insome implementations, the autonomous vehicle can obtain the locationdata using multiple input, multiple output communication between theautonomous vehicle and a user device associated with the user. This canallow the autonomous vehicle to take advantage of the multiple antennasincluded in the vehicle's communication system as well as those of theuser device to increase accuracy of the location data associated withthe user. In some implementations, the estimated time to user arrivalcan be based at least in part on historic data. Such historic data caninclude, for example, previous correlations between changes in thesignal strength of an identifier and the user's time to arriving at avehicle.

Additionally, or alternatively, the autonomous vehicle, systems, andmethods described herein can utilize other communication methods. Suchcommunication methods can include, for example, autonomous vehicle basedtriangulation (e.g., on vehicle triangulated RF), on-vehicle multi-rangebeacon (e.g., Bluetooth Low Energy) hardware (e.g., paired with asoftware application on a user device), application triangulation, userdevice to vehicle handshake (e.g., light signal handshake), autonomousvehicle perception of the user (e.g., via processing of sensor data toperceive the user and the user's location, distance, heading, velocity,and/or other state data associated therewith), GPS location of the userdevice, device specific techniques (e.g., associated with a specificdevice/model type), the autonomous vehicle serving as a localized basestation (e.g., GPS, WiFi, etc.) for the user device, autonomous vehiclelocalization and user device image based localization via one or morenetwork(s), and/or other techniques.

In some implementations, in order to determine whether the amount ofstopping time is acceptable, the autonomous vehicle can compare theestimated time of user arrival to a time constraint. The time constraintcan be expressed as a time threshold (e.g., indicating an acceptableamount of stopping time) and/or cost data (e.g., cost functionsexpressing a cost in relation to stopping time). Similar to the trafficconstraint, the time constraint can be based on historic data (e.g.,indicating historic wait times), real-time data (e.g., indicating thatthe vehicles are already waiting due to another traffic build-up infront of the autonomous vehicle), expectations of individuals in thegeographic area, machine-learned model(s), and/or other information.

In some implementations, the estimated time of user arrival can alsofactor in additional amounts of time that can impact the object(s)within the surrounding environment of the autonomous vehicle while theautonomous vehicle is stopped, awaiting the user. Example instancesrequiring such additional amounts of time can be associated with a usergetting into the vehicle and securely fastening his/her seatbelt, a userhelping other passengers enter and become securely positioned within thevehicle (e.g., children or others requiring assistance), a user loadingluggage or other items for transportation within the vehicle, a userreceiving delivered item(s) from/placing item(s) within the vehicle,etc. In some implementations, the additional amounts of time can bedetermined based at least in part on information provided with a servicerequest (e.g., type of service, destination, number of passengers,child's car seat requested, etc.).

The autonomous vehicle can determine one or more vehicle actions basedat least in part on the estimated traffic impact and/or the estimatedtime of user arrival. In some implementations, vehicle action(s) caninclude stopping within the vicinity of the location associated with theuser (e.g., at least partially in the travel way). For example, in theevent that the level of traffic (e.g., the number of other vehicles thatwould be impacted by an in-lane stop) is below a traffic threshold, theautonomous vehicle can determine that the vehicle can stop within thetravel way to wait for the user to arrive at the vehicle. In anotherexample, in the event that the estimated time of user arrival is belowthe time threshold, the autonomous vehicle can determine that thevehicle can stop at least partially within the travel way.

In some implementations, the autonomous vehicle can base itsdetermination to stop at least partially within the travel way on boththe estimated traffic impact and the estimated time of user arrival. Forinstance, the autonomous vehicle can weigh each of these estimates todetermine whether it would be appropriate for the vehicle to stop atleast partially in the travel way to wait for the user. The autonomousvehicle can apply a first weighting factor to the estimated trafficimpact and a second weighting factor to the estimated time of userarrival. The first weighting factor can be different than the secondweighting factor. By way of example, the estimated impact on traffic maybe high while the estimated time of user arrival may be short.Accordingly, the autonomous vehicle may determine that it can stopwithin the travel way because although a higher number of objects (e.g.,other vehicles) may be caused to stop, it would only be for a shortperiod of time because the user is close in distance to the autonomousvehicle. In such a case, the estimated time of user arrival can be givena greater weight than the estimated traffic impact. In another example,the estimated impact on traffic may be low while the estimated time ofuser arrival may be long. Accordingly, the autonomous vehicle maydetermine that it should not stop within the travel way because althoughonly a few objects (e.g., other vehicles) may be caused to stop, itwould be for a greater period of time because the user is farther fromthe autonomous vehicle. In such a case, the estimated traffic impact canbe given a greater weight than the estimated time of user arrival.

The vehicle action(s) can also, or alternatively, include the autonomousvehicle entering into a holding pattern. In such a case, the vehicleaction(s) can include traveling along a second vehicle route (e.g., anoptimal holding pattern route). For example, the autonomous vehicle maybe unable to find a parking location before and/or after the locationassociated with the user (e.g., before and/or after a pin location ofthe user). Additionally, the autonomous vehicle may determine that itshould not stop within a travel way to wait for the user's arrival, asdescribed herein. Thus, the autonomous vehicle can be re-routed along asecond vehicle route that is at least partially different than the firstvehicle route. The second vehicle route can be a path along which theautonomous vehicle can travel to re-arrive within the vicinity of thelocation of the user. For example, the second vehicle route can be apath along which the autonomous vehicle can travel around a block, backto the location associated with the user. This can afford the useradditional time to arrive at the vehicle, without the autonomous vehicleimpacting traffic (e.g., due to a stop).

In some implementations, the autonomous vehicle can determine that itcan stop within the travel way, but later determine that it must beginto travel again (e.g., according to a holding pattern route). Forexample, the autonomous vehicle can determine that it would beappropriate to stop at least partially within the travel way to wait forthe user based at least in part on the estimated traffic impact and/orthe estimated time of user arrival. While the autonomous vehicle isstopped, the traffic impact may increase (e.g., due to an increase inthe number of other vehicle(s) stopped behind the autonomous vehicle)and/or the user may take longer than estimated to arrive at theautonomous vehicle. As such, the autonomous vehicle can determine anupdated estimated traffic impact (e.g., based on the number of vehiclesthat have already stopped and/or additional vehicles that may be causedto stop) and/or an updated estimated time of user arrival (e.g., basedon a change in the user device location data, if any). The autonomousvehicle can then determine that it can no longer remain stopped to waitfor the user based at least in part on the updated estimates. As such,the autonomous vehicle can begin to travel again, for example, along thesecond vehicle route (e.g., around the block).

The vehicle computing system of the autonomous vehicle can implement thedetermined vehicle action(s). For example, in the event that theautonomous vehicle has determined to stop in the travel way, the vehiclecomputing system can cause the autonomous vehicle to stop by sending oneor more control signals to the braking control system(s) of theautonomous vehicle. In the event that the autonomous vehicle hasdetermined to travel along a second vehicle route (e.g., in accordancewith the holding pattern), the vehicle computing system can obtain dataassociated with the second vehicle route and implement the routeaccordingly. For example, the vehicle computing system can request andobtain data indicative of the second vehicle route from the operationscomputing system. Additionally, or alternatively, the vehicle computingsystem can determine the second vehicle route onboard the vehicle. Ineither case, the vehicle computing system can send one or more controlsignals to cause a motion planning system of the autonomous vehicle toplan the motion of the vehicle in accordance with the second vehicleroute (e.g., to implement a vehicle trajectory in accordance with thesecond vehicle action).

The autonomous vehicle can provide the user with one or morecommunications indicating the actions taken by the autonomous vehicle.For example, in the event that the autonomous vehicle stops within thetravel way, the autonomous vehicle can provide a communication to theuser indicating that the vehicle is waiting for the user (e.g., “yourvehicle has arrived, please proceed quickly to the vehicle”). Inresponse to receiving such a communication, a user device associatedwith the user can display a map user interface indicating the vehiclelocation of the autonomous vehicle and a user route to the vehicle'slocation. In the event that the autonomous vehicle does not find aparking spot and does not stop in the travel way, the autonomous vehiclecan provide a communication to the user indicating as such (e.g., “Icould not locate you at the pin drop, traffic forced me to go around theblock. Please proceed to the pin drop”).

In some implementations, the autonomous vehicle may be relieved of itsresponsibility to provide a vehicle service to the user. For instance,in the event that a vehicle computing system and/or operations computingsystem determines that an autonomous vehicle should travel along thesecond vehicle route, such computing system(s) can determine whether itwould be advantageous (e.g., more time efficient, more fuel efficient,etc.) for another autonomous vehicle in the area to be routed to theuser. In the event that it would be advantageous, the computingsystem(s) can provide data to the autonomous vehicle indicating that theautonomous vehicle is no longer responsible for the service request.Additionally, or alternatively, the computing system(s) can re-route theautonomous vehicle to provide a vehicle service to another user.

In some implementations, the autonomous vehicle can cancel the servicerequest of the user. By way of example, the autonomous vehicle may becaused to re-route (e.g., circle the block) a certain number of timesand/or the user may not arrive at the autonomous vehicle within acertain timeframe (e.g., above a trip cancellation threshold). Inresponse, the autonomous vehicle can send data to the operationscomputing system requesting the cancellation of the user's servicerequest. The operations computing system can cancel the service request(and inform the user accordingly) and/or re-route the autonomous vehicleto provide a vehicle service to another user. In some implementations,the autonomous vehicle can communicate directly with a user deviceassociated with the user to cancel the service request and inform theuser accordingly. The autonomous vehicle can report such a cancellationto the operations computing system.

The systems and methods described herein may provide a number oftechnical effects and benefits. For instance, the systems and methodsenable an autonomous vehicle to determine a holding pattern, for waitingfor a user, onboard the autonomous vehicle. As such, the autonomousvehicle need not communicate with a remote computing system (e.g.,operations computing system) each time the vehicle must decide whetherto stop in a travel way or re-route the vehicle (e.g., around theblock). This can help improve the response time of the vehicle computingsystem when deciding and/or implementing such a holding pattern.Moreover, by enabling the vehicle computing system to determine whetherto stop and/or to re-route onboard the vehicle, the systems and methodsdescribed herein can save computational resources of the operationscomputing system (that would otherwise be required for suchdetermination). Accordingly, the computational resources of theoperations computing system can be allocated to other core functionssuch as the management of service requests, routing of autonomousvehicles, etc.

The systems and methods of the present disclosure also provide animprovement to vehicle computing technology, such as autonomous vehiclecomputing technology. For instance, the computer-implemented methods andsystems improve the situational awareness of the autonomous vehicle toprovide a vehicle service to a user. The systems and methods can enablea computing system to obtain data indicative of a location associatedwith a user to which an autonomous vehicle is to travel. The autonomousvehicle can travel along a first vehicle route that leads to thelocation associated with the user. The computing system can obtaintraffic data associated with a geographic area that includes thelocation associated with the user and location data associated with auser device associated with the user. The computing system can determinean estimated traffic impact of the autonomous vehicle on the geographicarea based at least in part on the traffic data and/or an estimated timeof user arrival based at least in part on the location data associatedwith the user device. The computing system can determine one or morevehicle actions based at least in part on the estimated traffic impactand/or the estimated time of user arrival. The computing system cancause the autonomous vehicle to perform the one or more vehicle actions.As described herein, the vehicle action(s) can include at least one ofstopping the autonomous vehicle at least partially in a travel waywithin a vicinity of the location associated with the user or travellingalong a second vehicle route. In this way, the vehicle computing systemcan improve the holding pattern of the autonomous vehicle that iswaiting for a user. For example, the holding pattern can be customizedbased on the estimated impact on the traffic surrounding the autonomousvehicle and/or the estimated time it will take for the specific user toarrive at the vehicle. As such, the systems and methods can improve thevehicle computing system's situational awareness by allowing it to takeinto account (e.g., in real-time) such circumstances when making adetermination as to how to best provide vehicle services. Such approachcan also increase the efficiency of implementing a holding pattern(e.g., by avoiding the aforementioned latency issues) while providing anadditional benefit of minimizing the autonomous vehicle's impact ontraffic.

Additionally, the systems and methods of the present disclosure canenhance the user experience associated with the autonomous vehicle. Forinstance, the systems and methods described herein provide a systematicapproach that enables users to engage autonomous vehicles and receiveeffectively communicated information regarding expected autonomouslocations including, for example, arrival times, use of holding patternswhen needed, etc. By way of example, the communications and userinterfaces described herein provide the user with updated informationregarding the autonomous vehicle's actions and locations, therebyincreasing the user's knowledge and understanding of the autonomousvehicle's intentions. In addition to these advantages, the userexperience is further enhanced in that the described systems and methodscan decrease the likelihood that a user will be subjected to potentialfrustration from other drivers or pedestrians in the surroundingenvironment that could be impacted while the user is arriving to and/orboarding the autonomous vehicle.

With reference now to the FIGS., example embodiments of the presentdisclosure will be discussed in further detail. FIG. 1 depicts anexample system 100 according to example embodiments of the presentdisclosure. The system 100 can include a vehicle computing system 102associated with a vehicle 104 and an operations computing system 106that is remote from the vehicle 104.

The vehicle 104 incorporating the vehicle computing system 102 can be aground-based autonomous vehicle (e.g., car, truck, bus, etc.), anair-based autonomous vehicle (e.g., airplane, drone, helicopter, orother aircraft), or other types of vehicles (e.g., watercraft, etc.).The vehicle 104 can be an autonomous vehicle that can drive, navigate,operate, etc. with minimal and/or no interaction from a human driver. Ahuman operator can be omitted from the vehicle 104 (and/or also omittedfrom remote control of the vehicle 104).

The vehicle 104 can be configured to operate in a plurality of operatingmodes 108A-D. The vehicle 104 can be configured to operate in a fullyautonomous (e.g., self-driving) operating mode 108A in which the vehicle104 can drive and navigate with no input from a user present in thevehicle 104. The vehicle 104 can be configured to operate in asemi-autonomous operating mode 108B in which the vehicle 104 can operatewith some input from a user present in the vehicle 104. The vehicle 104can enter into a manual operating mode 108C in which the vehicle 104 isfully controllable by a user (e.g., human driver) and can be prohibitedfrom performing autonomous navigation (e.g., autonomous driving). Insome implementations, the vehicle 104 can implement vehicle operatingassistance technology (e.g., collision mitigation system, power assiststeering, etc.) while in the manual operating mode 108C to help assistthe operator of the vehicle 104.

The vehicle 104 can also be configured to operate in an approach mode108D in which vehicle 104 performs various functions as it approaches alocation associated with a user. For example, the vehicle 104 can enterinto the approach mode 108D when the vehicle is within a vicinity of auser to which the vehicle 104 is to provide a vehicle service, asfurther described herein. While in the approach mode 108D, the vehicle104 can search its surrounding environment for a parking location aswell as communicate with a user (e.g., via a user device associated withthe user). Additionally, or alternatively, the vehicle 104 can beconfigured to evaluate its estimated impact on traffic and/or anestimated time of user arrival, as further described herein, when thevehicle 104 is in the approach mode 108D.

The operating mode of the vehicle 104 can be adjusted in a variety ofmanners. In some implementations, the operating mode of the vehicle 104can be selected remotely, off-board the vehicle 104. For example, anentity associated with the vehicle 104 (e.g., a service provider) canutilize the operations computing system 106 to manage the vehicle 104(and/or an associated fleet). The operations computing system 106 cansend one or more control signals to the vehicle 104 instructing thevehicle 104 to enter into, exit from, maintain, etc. an operating mode.By way of example, the operations computing system 106 can send one ormore control signals to the vehicle 104 instructing the vehicle 104 toenter into the fully autonomous operating mode 108A. In someimplementations, the operating mode of the vehicle 104 can be setonboard and/or near the vehicle 104. For example, the vehicle computingsystem 102 can automatically determine when and where the vehicle 104 isto enter, change, maintain, etc. a particular operating mode (e.g.,without user input). Additionally, or alternatively, the operating modeof the vehicle 104 can be manually selected via one or more interfaceslocated onboard the vehicle 104 (e.g., key switch, button, etc.) and/orassociated with a computing device proximate to the vehicle 104 (e.g., atablet operated by authorized personnel located near the vehicle 104).In some implementations, the operating mode of the vehicle 104 can beadjusted based at least in part on a sequence of interfaces located onthe vehicle 104. For example, the operating mode may be adjusted bymanipulating a series of interfaces in a particular order to cause thevehicle 104 to enter into a particular operating mode.

The vehicle computing system 102 can include one or more computingdevices located onboard the vehicle 104. For example, the computingdevice(s) can be located on and/or within the vehicle 104. The computingdevice(s) can include various components for performing variousoperations and functions. For instance, the computing device(s) caninclude one or more processor(s) and one or more tangible,non-transitory, computer readable media (e.g., memory devices). The oneor more tangible, non-transitory, computer readable media can storeinstructions that when executed by the one or more processor(s) causethe vehicle 104 (e.g., its computing system, one or more processors,etc.) to perform operations and functions, such as those describedherein for controlling an autonomous vehicle.

As shown in FIG. 1, the vehicle 104 can include one or more sensors 112,an autonomy computing system 114, and one or more vehicle controlsystems 116. One or more of these systems can be configured tocommunicate with one another via a communication channel. Thecommunication channel can include one or more data buses (e.g.,controller area network (CAN)), on-board diagnostics connector (e.g.,OBD-II), and/or a combination of wired and/or wireless communicationlinks. The onboard systems can send and/or receive data, messages,signals, etc. amongst one another via the communication channel.

The sensor(s) 112 can be configured to acquire sensor data 118associated with one or more objects that are proximate to the vehicle104 (e.g., within a field of view of one or more of the sensor(s) 112).The sensor(s) 112 can include a Light Detection and Ranging (LIDAR)system, a Radio Detection and Ranging (RADAR) system, one or morecameras (e.g., visible spectrum cameras, infrared cameras, etc.), motionsensors, and/or other types of imaging capture devices and/or sensors.The sensor data 118 can include image data, radar data, LIDAR data,and/or other data acquired by the sensor(s) 112. The object(s) caninclude, for example, pedestrians, vehicles, bicycles, and/or otherobjects. The object(s) can be located in front of, to the rear of,and/or to the side of the vehicle 104. The sensor data 118 can beindicative of locations associated with the object(s) within thesurrounding environment of the vehicle 104 at one or more times. Thesensor(s) 112 can provide the sensor data 118 to the autonomy computingsystem 114.

In addition to the sensor data 118, the autonomy computing system 114can retrieve or otherwise obtain map data 120. The map data 120 canprovide detailed information about the surrounding environment of thevehicle 104. For example, the map data 120 can provide informationregarding: the identity and location of different roadways, roadsegments, buildings, or other items or objects (e.g., lampposts,crosswalks, curbing, etc.); the location and directions of traffic lanes(e.g., the location and direction of a parking lane, a turning lane, abicycle lane, or other lanes within a particular roadway or other travelway and/or one or more boundary markings associated therewith); trafficcontrol data (e.g., the location and instructions of signage, trafficlights, or other traffic control devices); and/or any other map datathat provides information that assists the vehicle 104 in comprehendingand perceiving its surrounding environment and its relationship thereto.

The vehicle 104 can include a positioning system 122. The positioningsystem 122 can determine a current position of the vehicle 104. Thepositioning system 122 can be any device or circuitry for analyzing theposition of the vehicle 104. For example, the positioning system 122 candetermine position by using one or more of inertial sensors, a satellitepositioning system, based on IP/MAC address, by using triangulationand/or proximity to network access points or other network components(e.g., cellular towers, WiFi access points, etc.) and/or other suitabletechniques. The position of the vehicle 104 can be used by varioussystems of the vehicle computing system 102 and/or provided to one ormore remote computing device(s) (e.g., of the operations computingsystem 106). For example, the map data 120 can provide the vehicle 104relative positions of the surrounding environment of the vehicle 104.The vehicle 104 can identify its position within the surroundingenvironment (e.g., across six axes) based at least in part on the datadescribed herein. For example, the vehicle 104 can process the sensordata 118 (e.g., LIDAR data, camera data) to match it to a map of thesurrounding environment to get an understanding of the vehicle'sposition within that environment.

The autonomy computing system 114 can include a perception system 124, aprediction system 126, a motion planning system 128, and/or othersystems that cooperate to perceive the surrounding environment of thevehicle 104 and determine a motion plan for controlling the motion ofthe vehicle 104 accordingly. For example, the autonomy computing system114 can receive the sensor data 118 from the sensor(s) 112, attempt tocomprehend the surrounding environment by performing various processingtechniques on the sensor data 118 (and/or other data), and generate anappropriate motion plan through such surrounding environment. Theautonomy computing system 114 can control the one or more vehiclecontrol systems 116 to operate the vehicle 104 according to the motionplan.

The autonomy computing system 114 can identify one or more objects thatare proximate to the vehicle 104 based at least in part on the sensordata 118 and/or the map data 120. For example, the perception system 124can obtain state data 130 descriptive of a current state of an objectthat is proximate to the vehicle 104. The state data 130 for each objectcan describe, for example, an estimate of the object's: current location(also referred to as position); current speed (also referred to asvelocity); current acceleration; current heading; current orientation;size/footprint (e.g., as represented by a bounding shape); class (e.g.,pedestrian class vs. vehicle class vs. bicycle class), and/or otherstate information. The perception system 124 can provide the state data130 to the prediction system 126 (e.g., for predicting the movement ofan object).

The prediction system 126 can create predicted data 132 associated witheach of the respective one or more objects proximate to the vehicle 104.The predicted data 132 can be indicative of one or more predicted futurelocations of each respective object. The predicted data 132 can beindicative of a predicted path (e.g., predicted trajectory) of at leastone object within the surrounding environment of the vehicle 104. Forexample, the predicted path (e.g., trajectory) can indicate a path alongwhich the respective object is predicted to travel over time (and/or thespeed at which the object is predicted to travel along the predictedpath). The prediction system 126 can provide the predicted data 132associated with the object(s) to the motion planning system 128.

The motion planning system 128 can determine a motion plan 134 for thevehicle 104 based at least in part on the predicted data 132 (and/orother data). The motion plan 134 can include vehicle actions withrespect to the objects proximate to the vehicle 104 as well as thepredicted movements. For instance, the motion planning system 128 canimplement an optimization algorithm that considers cost data associatedwith a vehicle action as well as other objective functions (e.g., costfunctions based on speed limits, traffic lights, etc.), if any, todetermine optimized variables that make up the motion plan 134. By wayof example, the motion planning system 128 can determine that thevehicle 104 can perform a certain action (e.g., pass an object) withoutincreasing the potential risk to the vehicle 104 and/or violating anytraffic laws (e.g., speed limits, lane boundaries, signage). The motionplan 134 can include a planned trajectory, speed, acceleration, otheractions, etc. of the vehicle 104.

The motion planning system 128 can provide the motion plan 134 with dataindicative of the vehicle actions, a planned trajectory, and/or otheroperating parameters to the vehicle control system(s) 116 to implementthe motion plan 134 for the vehicle 104. For instance, the vehicle 104can include a mobility controller configured to translate the motionplan 134 into instructions. By way of example, the mobility controllercan translate a determined motion plan 134 into instructions to adjustthe steering of the vehicle 104 “X” degrees, apply a certain magnitudeof braking force, etc. The mobility controller can send one or morecontrol signals to the responsible vehicle control component (e.g.,braking control system, steering control system, acceleration controlsystem) to execute the instructions and implement the motion plan 134.

The vehicle 104 can include a communications system 136 configured toallow the vehicle computing system 102 (and its computing device(s)) tocommunicate with other computing devices. The vehicle computing system102 can use the communications system 136 to communicate with theoperations computing system 106 and/or one or more other remotecomputing device(s) over one or more networks (e.g., via one or morewireless signal connections). In some implementations, thecommunications system 136 can allow communication among one or more ofthe system(s) on-board the vehicle 104. The communications system 136can also be configured to enable the autonomous vehicle to communicationand/or otherwise receive data from a user device 138 associated with auser 110. The communications system 136 can utilize variouscommunication technologies such as, for example, Bluetooth low energyprotocol, radio frequency signaling, etc. In some implementations, thecommunications systems 136 can enable the vehicle 104 to function as aWiFi base station for a user device 138 and/or implement localizationtechniques. The communications system 136 can include any suitablecomponents for interfacing with one or more network(s), including, forexample, transmitters, receivers, ports, controllers, antennas, and/orother suitable components that can help facilitate communication.

The vehicle 104 can include one or more human-machine interfaces 139.For example, the vehicle 104 can include one or more display deviceslocated onboard the vehicle 104. A display device (e.g., screen of atablet, laptop, etc.) can be viewable by a user of the vehicle 104 thatis located in the front of the vehicle 104 (e.g., driver's seat, frontpassenger seat). Additionally, or alternatively, a display device can beviewable by a user of the vehicle 104 that is located in the rear of thevehicle 104 (e.g., back passenger seat(s)).

In some implementations, the vehicle 104 can be associated with anentity (e.g., a service provider, owner, manager). The entity can be onethat provides one or more vehicle service(s) to a plurality of users viaa fleet of vehicles that includes, for example, the vehicle 104. In someimplementations, the entity can be associated with only vehicle 104(e.g., a sole owner, manager). In some implementations, the operationscomputing system 106 can be associated with the entity. The vehicle 104can be configured to provide one or more vehicle services to one or moreusers. The vehicle service(s) can include transportation services (e.g.,rideshare services in which the user rides in the vehicle 104 to betransported), courier services, delivery services, and/or other types ofservices. The vehicle service(s) can be offered to users by the entity,for example, via a software application (e.g., a mobile phone softwareapplication). The entity can utilize the operations computing system 106to coordinate and/or manage the vehicle 104 (and its associated fleet,if any) to provide the vehicle services to a user 110.

The operations computing system 106 can include one or more computingdevices that are remote from the vehicle 104 (e.g., located off-boardthe vehicle 104). For example, such computing device(s) can becomponents of a cloud-based server system and/or other type of computingsystem that can communicate with the vehicle computing system 102 of thevehicle 104. The computing device(s) of the operations computing system106 can include various components for performing various operations andfunctions. For instance, the computing device(s) can include one or moreprocessor(s) and one or more tangible, non-transitory, computer readablemedia (e.g., memory devices). The one or more tangible, non-transitory,computer readable media can store instructions that when executed by theone or more processor(s) cause the operations computing system 106(e.g., the one or more processors, etc.) to perform operations andfunctions, such as coordinating vehicles to provide vehicle services.

For example, a user 110 can request a vehicle service provided by thevehicle 104. For instance, a user can provide (e.g., via a user device138) data indicative of a request 140 to the operations computing system106 (e.g., of the entity that is associated with the vehicle 104). Insome implementations, the request 140 can be generated based at least inpart on user input to a user interface displayed on the user device 138(e.g., a user interface associated with a software application of theentity). The request 140 can indicate the type of vehicle service thatthe user 110 desires (e.g., a transportation service, a deliveryservice, a courier service, etc.) and a location associated with theuser 110 (e.g., a current location of the user, a different location,etc.). The request 140 can also include an identifier (e.g., phonenumber, Bluetooth, WiFi, Cellular, IP address, other information, etc.)associated with the user device 138 that provided the request 140(and/or other user device). The identifier can be used by the vehiclecomputing system 102 to communicate with the user device 138 and/orotherwise provide/obtain data associated therewith, as further describedherein. In some implementations, such an identifier can be retrievedfrom a memory that securely stores such information in a profile/accountassociated with the user 110 (e.g., such that the request 140 need notprovide the identifier).

The operations computing system 106 can process the request 140 andselect the vehicle 104 to provide the requested vehicle service to theuser 110. The operations computing system can provide, to the vehicle104, data 142 indicative of a location to which the vehicle 104 is totravel. For example, FIG. 2 depicts an example geographic area 200 thatincludes a location 202 associated with the user 110 according toexample embodiments of the present disclosure. The location 202 can beassociated with the user 110 that requested the vehicle service. Forexample, the location 202 can be the current location of the user 110,as specified by the user 110 and/or determined based on user devicelocation data (e.g., provided with the request 140 and/or otherwiseobtained). The location 202 can also be a location that is differentthan a current location of a user 110, such as for example a location atwhich the user 110 would like to be picked-up by the vehicle 104,provide an item to the vehicle 104, retrieve an item from the vehicle104, and/or otherwise interact with the vehicle 104. The location 202can be expressed as a coordinate (e.g., GPS coordinate,latitude-longitude coordinate pair), an address, a place name, and/oranother geographic reference that can be used to identify the location202. The location 202 associated with the user 110 can be represented,for example, as a pin on a map user interface.

The vehicle computing system 102 can obtain the data 142 indicative ofthe location 202 associated with the user 110 to which the vehicle 104is to travel. As described herein, the user 110 can be associated with arequest 140 for a vehicle service provided by the vehicle 104. Thevehicle 104 can obtain the data 142 indicative of the location 202associated with the user 110 from the operations computing system 106.In some implementations, the user 110 may communicate directly with thevehicle 104 to request the vehicle service. For example, the user 110may use the user device 138 to send the request 140 to the vehiclecomputing system 102. In such a case, the vehicle computing system 102can process the request 140 and determine the location 202 of the user110.

The vehicle 104 can also obtain a first vehicle route 204 that leads tothe location 202 associated with the user 110. The first vehicle route204 can be, for example, a route from the current location of thevehicle 104 to the location 202 associated with the user 110. In someimplementations, the operations computing system 106 can provide thefirst vehicle route 204 to the vehicle 104. Additionally, oralternatively, the vehicle computing system 102 of the vehicle 104 candetermine the first vehicle route 204. For example, the vehiclecomputing system 102 can determine the first vehicle route 204 based atleast in part on the map data 120.

The vehicle 104 is to travel in accordance with the first vehicle route204 to arrive within a vicinity 206 of the location 202 associated withthe user 110. For example, the vehicle computing system 102 can controlthe vehicle 104 (e.g., via a motion plan 134 implemented by the controlsystem(s) 116) to travel along a first vehicle route 204 to arrivewithin a vicinity 206 of the location 202 associated with the user 110.The vicinity 206 of the location 202 associated with the user 110 can bedefined at least in part by a distance (e.g., a radial distance) fromthe location 202 associated with the user 110. The distance can beindicative of an acceptable walking distance from the location 202associated with the user 110. For example, an acceptable walkingdistance can be distance that a user would be willing walk (or otherwisetravel) to arrive at a vehicle. In some implementations, the operationscomputing system 106 can determine the acceptable walking distance, asdescribed herein, and provide such information to the vehicle 104. Insome implementations, the vehicle 104 can determine the acceptablewalking distance. In some implementations, the acceptable walkingdistance can be determined by another computing system and provided tothe operations computing system 106 and/or the vehicle 104.

The acceptable walking distance can be determined based on a variety ofinformation. FIG. 3 depicts example information 300 associated with anacceptable walking distance according to example embodiments of thepresent disclosure. The acceptable walking distance 302 can bedetermined based at least in part on at least one of current trafficdata 304A, historic data 304B, user preference data 304C, local vehicleweather data 304D, regional weather data 304E, and/or other data. Thecurrent traffic data 304A can be indicative of a current level oftraffic within the geographic area 200 and/or an area that would affectthe geographic area 200. The current traffic data 304A can be obtainedfrom another computing system (e.g., city management database, theoperations computing system 106, another vehicle, etc.) and/ordetermined by a vehicle 104, as further described herein. For example,in the event that the current traffic level is high such that it wouldtake a longer time for the vehicle 104 to reach the location 202, theacceptable walking distance 302 may be higher so that the user 110 isn'twaiting a greater time to board the vehicle 104, place an item in thevehicle 104, retrieve an item from the vehicle, etc.

The historic data 304B can include historic data associated withproviding vehicle services to a user. For instance, the historic data304B can be indicative of previously calculated acceptable walkingdistances for the specific user 110 and/or for other user(s) of thevehicle services. By way of example, in the event that the user (orsimilarly situated user(s)) has shown a willingness to walk a certaindistance to a vehicle, the acceptable walking distance 302 can bedetermined to reflect the historically acceptable walking distance. Insome implementations, the historic data 304B can include historictraffic data.

Additionally, or alternatively, the acceptable walking distance 302 canbe based at least in part on preferences of the user 110. For instance,the entity associated with the vehicle 104 can maintain anaccount/profile associated with the user 110. In some implementations,the user 110 can specify an acceptable walking distance (e.g., via userinput to a user interface). The user-specified acceptable walkingdistance can be securely stored and used to determine the acceptablewalking distance 302 when the user 110 requests a vehicle service. Insome implementations, the user 110 may provide feedback regarding thedistance the user 110 walked to arrive at a vehicle that is providingthe user 110 vehicle services. The user 110 can provide feedback data(e.g., via user input to a user interface) indicating whether thedistance was acceptable or unacceptable (e.g., as prompted by a softwareapplication). Accordingly, the acceptable walking distance 302 can bebased at least in part on such feedback data. In some implementations,acceptable walking distance 302 can be based at least in part onpreferences of other users such as those similar situated to the user110, within the geographic area, within a similar geographic area, etc.

The acceptable walking distance 302 can also, or alternatively, be basedat least in part on weather data. In some implementations, theacceptable walking distance 302 can be based at least in part on localweather data 304D obtained via a vehicle. For example, the vehicle 104can include a rain sensor, thermometer, humidity sensor, and/or othertypes of sensor(s) that can be used to determine weather conditionswithin the surrounding environment of the vehicle 104. The vehicle 104can also be configured to determine the presence of one or more weatherconditions (e.g., rain, sleet, snow, etc.) based at least in part on thesensor data 118. Additionally, or alternatively, the acceptable walkingdistance 302 can be based at least in part on regional weather data 304E(e.g., from a third party weather source). The acceptable walkingdistance 302 can be adjusted depending on the weather conditionsindicated by the weather data. By way of example, in the event that thelocal weather data 304D and/or the regional weather data 304E indicatethat the geographic area 200 is experiencing (and/or will experience)rain, the acceptable walking distance 302 may be decreased to a shortdistance from the location 202. In the event that the local weather data304D and/or the regional weather data 304E indicate that the geographicarea 200 is experiencing (and/or will experience) clear skies, theacceptable walking distance 302 can be greater distance from thelocation 202.

In some implementations, the acceptable walking distance 302 can bedetermined at least in part from a model, such as a machine-learnedmodel. For example, the machine-learned model can be or can otherwiseinclude one or more various model(s) such as, for example, models usingboosted random forest techniques, neural networks (e.g., deep neuralnetworks), or other multi-layer non-linear models. Neural networks caninclude recurrent neural networks (e.g., long short-term memoryrecurrent neural networks), feed-forward neural networks, and/or otherforms of neural networks. For instance, supervised training techniquescan be performed to train the model using historical acceptable walkingdistances, weather data, user feedback data, etc. to determine anacceptable walking distance 302 based at least in part on input data.The input data can include, for example, the various types ofinformation 300, as described herein. The machine-learned model canprovide, as an output, data indicative of an acceptable walking distance302. In some implementations, the acceptable walking distance 302 can bespecific to the geographic area. For example, the model can be trainedbased on information associated with the geographic area 200 such thatthe outputted acceptable walking distance 302 is specific to thegeographic area 200.

In some implementations, the vehicle 104 can provide a communication tothe user 110 indicating that the vehicle 104 is within the vicinity 206of the location 202 associated with the user 110. For example, thevehicle computing system 102 can determine that the vehicle 104 iswithin the acceptable walking distance 302 from the location 202 of theuser 110 (e.g., based on the positioning system 144). In response, thevehicle computing system 102 can send a communication to the user device138 associated with the user 110. The communication can indicate thatthe vehicle 104 is within the vicinity 206 of the user 110 (e.g., withinthe acceptable walking distance 302). For instance, for a transportationservice, such a communication can be in the form of a textual message,auditory message, etc. stating “your vehicle is arriving please prepareto board”.

The vehicle 104 can begin to search for a parking location (e.g., out ofthe vehicle's travel way) when the vehicle 104 is within the vicinity ofthe location 202 associated with the user 110. For example, the vehicle104 can enter into the approach operating mode 108D when the vehicle 104is within the vicinity 206 of the location 202 associated with the user110. While within the vicinity 206, the vehicle 104 can search for aparking location before it reaches the location 202 associated with theuser 110 (e.g., before the GPS pin coordinate on a map). In someimplementations, the vehicle 104 can search for a parking location afterit passes the location 202 associated with the user 110 and is stillwithin the vicinity 206 of the user 110 (e.g., within the acceptablewalking distance 302).

To identify a parking location, the vehicle computing system 102 canobtain sensor data 118 associated with one or more objects that areproximate to the vehicle 104. As described herein, the sensor data 118can be indicative of locations associated with the object(s) within thesurrounding environment of the vehicle 104 at one or more times. Thevehicle 104 can process the sensor data 118 to determine if there areany available parking locations out of the vehicle's travel way (e.g.,out of a traffic lane) that are not currently occupied by the objects(e.g., other vehicles) within the vehicle's surrounding environment. Insome implementations, the vehicle 104 can utilize the map data 120 todetermine whether any designated parking locations (e.g., parking lots,pullover lanes, etc.) are located and/or are available within thevicinity 206 of the location 202 associated with the user 110. In theevent the vehicle 104 can identify a parking location that is out of atravel way and within the vicinity 206 of the location 202 associatedwith the user 110 (e.g., out of a traffic lane), the vehicle 104 canposition itself into that parking location accordingly.

The vehicle computing system 102 can send a communication to the userdevice 138 associated with the user 110 indicating that the vehicle 104has parked. For example, FIG. 4 depicts an example display device 400with an example communication 402 according to example embodiments ofthe present disclosure. The display device 400 (e.g., display screen)can be associated with the user device 138 associated with the user 110.The communication 402 can be presented via a user interface 404 on thedisplay device 400. The communication 402 can indicate that the vehicle104 has arrived. In some implementations, the communication 402 and/oranother portion of the user interface 404 can be indicative of alocation of the vehicle 104. For example, the display device 400 candisplay a map user interface 406 that includes a user route 408. Theuser route 408 can be a route along which a user 110 can travel toarrive at the vehicle 104.

In the event that the vehicle 104 is unable to identify a parkinglocation that is out of the travel way (and within the vicinity 206 ofthe user 110), the vehicle 104 can decide whether or not to stop atleast partially in a travel way to wait for the user 110. To help do so,the vehicle computing system 102 of the vehicle 104 can obtain trafficdata associated with the geographic area 200 that includes the location202 associated with the user 110.

FIG. 5 depicts example traffic data 500 that can be obtained by thevehicle computing system 102 according to example embodiments of thepresent disclosure. The traffic data 500 can be obtained from a varietyof sources such as other vehicles (e.g., other autonomous vehicleswithin the vehicle fleet), the operations computing system 106, thirdparty sources (e.g., traffic management entities, etc.), as well as thevehicle 104 itself. The traffic data 500 can include, for example,in-lane traffic data 502A, out-of-lane traffic data 502B, other vehicletraffic data 502C, current wider traffic data 502D, historic trafficdata 502E, and/or other types of traffic data. The traffic data 500 canbe updated periodically, as scheduled, upon request, in real-time,and/or in near real-time.

The in-lane traffic data 502A and/or out-of-lane traffic data 502B canbe indicative of a level of traffic within the surrounding environmentof the vehicle 104. For instance, the in-lane traffic data 502A can beindicative of the number of objects, object locations, and the speed ofthe respective objects within the current travel lane (and/or otherdesignated travel boundaries) of the vehicle 104 (e.g., the othervehicles to the rear and front of the vehicle 104). The out-of-lanetraffic data 502B can be indicative of the number of objects, objectlocations, and the speed of the respective objects within thesurrounding environment, other than in the current travel lane (or otherboundaries) of the vehicle 104 (e.g., other vehicles in the other lanes,all other classified objects around the vehicle 104, etc.). The in-lanetraffic data 502A and/or out-of-lane traffic data 502B can be based atleast in part on the sensor data 118 associated with the surroundingenvironment of the vehicle 104, as further described herein.

The vehicle computing system 102 can obtain the other vehicle trafficdata 502C from one or more other vehicles, such as one or more otherautonomous vehicles in a fleet that includes the vehicle 104. The othervehicle traffic data 502C can be indicative of the number of objects,object locations, and the speed of the respective objects within thesurrounding environment of the other vehicle(s), while the othervehicle(s) are within the geographic area 200, the vicinity 206 of thelocation 202 associated with the user 110, and/or another location whichmay have a traffic effect on the geographic area 200 and/or the vicinity206 of the location 202 associated with the user 110. The vehiclecomputing system 102 can obtain the other vehicle traffic data 502C viavehicle to vehicle communication and/or via other computing device(s)(e.g., the operations computing system 106).

The current wider traffic data 502D can be indicative of the trafficwithin a larger regional area that includes the geographic area 200. Forinstance, the vehicle computing system 102 can obtain the current widertraffic data 502D that indicates the current traffic patterns,build-ups, etc. within a region that includes the geographic area 200.Such data can be obtained via a third party source such as, for example,a traffic management entity associated with the region.

The historic traffic data 502E can include previously collected trafficdata of the types of traffic data described herein and/or other types oftraffic data. For example, the historic traffic data 502E can includetraffic data previously obtained by the vehicle computing system 102(e.g., in-lane, out-of-lane traffic data), traffic data previouslyobtained by other vehicle(s) (e.g., associated with the geographic area200, the vicinity 206 of the location 202, etc.), historic traffic data(e.g., traffic patterns) associated with a region that includes thegeographic area 200, and/or other types of historic traffic data.

The vehicle computing system 102 can determine an estimated trafficimpact 504 of the vehicle 104 on the geographic area 200 based at leastin part on the traffic data 500. The estimated traffic impact 504 can beindicative of an estimated impact of the vehicle 104 on one or moreobjects within a surrounding environment of the vehicle 104 in the eventthat the vehicle 104 were to stop at least partially in the travel waywithin the vicinity 206 of the location 202 associated with the user110. In some implementations, the estimated traffic impact 504 canestimate the likelihood that an approach object (e.g., another vehicle)can pass safely without endangering the user 110.

In some implementations, the estimated traffic impact 504 can be basedat least in part on a comparison of a level of traffic to a trafficconstraint 506. The vehicle computing system 102 can determine a levelof traffic (e.g., a number of objects that could be impacted by thevehicle 104 stopping at least partially in a travel way) within thesurrounding environment of the vehicle 104 based at least in part on oneor more of the types of traffic data 500. The vehicle computing system102 can compare the level of traffic to a traffic constraint 506 todetermine whether the estimated traffic impact 504 would be high or low(e.g., significant or insignificant, unacceptable or acceptable, etc.).

The traffic constraint 506 can be implemented in a variety of forms. Forexample, the traffic constraint 506 can include a traffic threshold thatis indicative of an acceptable level of traffic (e.g., an acceptablenumber of objects) that would be impacted by the vehicle 104 stopping atleast partially in the travel way. A level of traffic that exceeds thetraffic threshold would be considered a high impact on traffic. In someimplementations, the traffic constraint 506 can be implemented as costdata (e.g., one or more cost function(s)). For example, the vehiclecomputing system 102 (e.g., the motion planning system 128) can includecost data that reflects the cost(s) of stopping vehicle motion, thecost(s) of causing traffic build-up, the cost(s) of illegally stoppingin a travel way, etc.

The traffic constraint 506 can be based on a variety of information. Insome implementations, the traffic constraint 506 can be based at leastin part on historic traffic data that indicates the level of trafficpreviously occurring in that geographic area. For example, if thegeographic area 200 normally experiences a high level of trafficbuild-up, a corresponding traffic threshold can be higher (and/or thecost of stopping can be lower). Additionally, or alternatively, thetraffic constraint 506 can be based at least in part on real-timetraffic data (e.g., from other vehicles in the fleet, from the vehicle104, other sources). For example, in the event that there is already atraffic jam in the vicinity 206 of the location 202 of the user 110, thetraffic threshold could be higher (and/or the cost of stopping could belower). In some implementations, the traffic constraint 506 can be basedat least in part on the typical travel expectations of individuals inthe geographic area 200. For example, individuals that are located inCity A may be more patient when waiting in traffic than those in City B.Thus, a traffic threshold may be higher in City A than in City B (and/orthe cost of stopping may be lower in City A than in City B). In someimplementations, the traffic constraint 506 can be based at least inpart on map data 120 (and/or other map data). For example, in the eventthat the vehicle 104 is traveling on a wide travel way in which impactedvehicles could eventually travel around the vehicle 104, the trafficthreshold could be higher (and/or the cost of stopping could be lower).The traffic constraint 506 can be determined dynamically, in real-time(and/or near real-time) to reflect the conditions currently faced by thevehicle 104.

In some implementations, the traffic constraint 506 can be determined atleast in part from a model, such as a machine-learned model. Forexample, the machine-learned model can be or can otherwise include oneor more various model(s) such as, for example, models using boostedrandom forest techniques, neural networks (e.g., deep neural networks),or other multi-layer non-linear models. Neural networks can includerecurrent neural networks (e.g., long short-term memory recurrent neuralnetworks), feed-forward neural networks, and/or other forms of neuralnetworks. For instance, supervised training techniques can be performedto train the model (e.g., using previous driving logs) to determine atraffic constraint 506 based at least in part on input data. The inputdata can include, for example, the traffic data 500 as described herein,map data, data from a traffic management entity, driving characteristicsof individuals in an associated geographic area, complaints receivedfrom operators of vehicles that were caused to stop by autonomousvehicles, etc. The machine-learned model can provide, as an output, dataindicative of a recommended traffic constraint. In some implementations,the recommended traffic constraint can be specific to the geographicarea 200. For example, the model can be trained based at least in parton data associated with geographic area 200 such that the recommendedtraffic constraint is specific for that particular region.

Additionally, or alternatively, the estimated traffic impact 504 can bebased at least in part on a model. The vehicle computing system 102and/or the operations computing system 106 can obtain data descriptiveof the model (e.g., machine learned model). The vehicle computing system102 and/or the operations computing system 106 can provide input data tothe model. The input data can include the one or more of the types oftraffic data 500 (e.g., associated with the geographic area 200). Insome implementations, the input data can include the map data 120. Themodel can determine the estimated traffic impact 504 that the vehicle104 would have on the geographic area 200 if the vehicle 104 were tostop at least partially within a travel way (e.g., within a current laneof travel). By way of example, the model can evaluate the traffic data500 to determine the level of traffic in the vehicle surroundingenvironment. The model can analyze the traffic data 500 with respect tothe traffic constraint 506 to determine whether the estimated trafficimpact 504 is high or low, significant or insignificant, unacceptable oracceptable, etc. The output of such a model can be the estimated trafficimpact 504, which can be indicative of, for example, a number of objectsthat would be impacted by the vehicle 104 stopping at least partially ina travel way and/or whether the estimated traffic impact is high or low,significant or insignificant, acceptable or unacceptable, etc.

In some implementations, the output of the model can be provided as aninput to the model for another set of traffic data (e.g., at asubsequent time step). In such fashion, confidence can be built that adetermined estimated traffic impact is the accurate. Stated differently,in some implementations, the process can be iterative such that theestimated traffic impact can be recalculated over time as it becomesclearer what the estimated traffic impact is on the respectivegeographic area. For example, the model can include one or moreautoregressive models. In some implementations, the model can includeone or more machine-learned recurrent neural networks. For example,recurrent neural networks can include long or short-term memoryrecurrent neural networks, gated recurrent unit networks, or other formsof recurrent neural networks.

As described herein, the estimated traffic impact 504 can be based atleast in part on the sensor data 118 acquired onboard the vehicle 104.For instance, the vehicle computing system 102 can obtain traffic dataassociated with the geographic area 200 that includes the location 202associated with the user 110 based at least in part on the sensor data118 obtained via the one or more sensors 112. By way of example, thevehicle computing system 102 can obtain the in-lane traffic data 502Aand/or out-of-lane traffic data 502B associated with the geographic area200 that includes the location 202 associated with the user 110. In someimplementations, the vehicle computing system 102 can obtain sensor data118 (e.g., via the one or more sensors 112) associated with thesurrounding environment of the vehicle 104 that is within the vicinity206 of the location 202 associated with the user 110, as describedherein. The sensor data 118 can be indicative of one or more objectswithin the surrounding environment of the vehicle 104 that is within thevicinity 206 of the location 202. The vehicle computing system 102 canprocess the sensor data 118 to classify which of the object(s) would beimpacted (e.g., caused to stop) by the vehicle 104 stopping in a travelway.

By way of example, FIG. 6 depicts an example travel way 600 according toexample embodiments of the present disclosure. The travel way 600 can beassociated with the geographic area 200. For example, the travel way 600can be located within the vicinity 206 of the location 202 associatedwith the user 110 (e.g., the street on which the user 110 is located).The travel way 600 can include the current travel lane 602 (and/or otherdesignated travel boundaries) of the vehicle 104. The travel way 600 caninclude other travel lanes, such as other lane 604 (e.g., a laneadjacent to the current travel lane 602). The vehicle computing system102 can classify the objects within the surrounding environment thatwould be impacted in the event the vehicle 104 were to stop at leastpartially in the travel way 600. For example, the object 606 (e.g.,another vehicle behind the vehicle 104 in the same travel lane 602) canbe identified as an object that would be impacted in the event that thevehicle 104 stops at least partially in the current lane 602.

The vehicle computing system 102 can also identify object(s) that wouldnot be affected by the vehicle 104 stopping at least partially in thetravel way 600. Such object(s) can include one or more objects inanother travel lane as well as objects with the same travel lane as thevehicle 104. For example, the vehicle computing system 102 can determinethat the object 608 located in the other travel lane 604 (e.g., anothervehicle in an adjacent travel lane) will not be impacted by the vehicle104 stopping at least partially in the travel way 600 because the object608 can continue passed the vehicle 104 via the other travel lane 604.In another example, the vehicle computing system 102 can determine thatan object 610 (e.g., a bicycle) may not be impacted by the vehicle 104stopping (and/or may be impacted to an insignificant degree) because theobject 608 may be parked and/or have the opportunity to travel aroundthe vehicle 104 (e.g., via a lane change into the adjacent lane,maneuver around the vehicle 104 within the same lane, etc.).

After such classification, the vehicle computing system 102 candetermine a level of traffic associated with the geographic area 200(e.g., within the vicinity of the user's location) based at least inpart on the sensor data 118. For example, the level of traffic can bebased at least in part on the number of object(s) within the surroundingenvironment of the vehicle 104 that would be impacted by the vehicle 104stopping at least partially in the travel way 600 (e.g., the currenttravel lane 602), while filtering out those object(s) that would not beimpacted. Similar such information could be acquired via one or moreother vehicles in the associated vehicle fleet. The vehicle computingsystem 102 can determine the estimated traffic impact by comparing thelevel of traffic to the traffic constraint 506 (e.g., a trafficthreshold indicative of a threshold level of traffic). In the event thatthe level of traffic exceeds the traffic constraint 506, the vehiclecomputing system 102 can determine that the estimated traffic impact 504on the geographic area 200 would be high.

The vehicle computing system 102 can also, or alternatively, determinean estimated time of user arrival. The estimated time of user arrivalcan be indicative of an amount of time needed for the user 110 tointeract with the vehicle 104 (e.g., before the vehicle 104 can beginmoving again). For example, the vehicle computing system 102 candetermined an estimated time until the user 110 completes boarding ofthe vehicle 104 (e.g., enters the vehicle 104 and fasten a seatbelt fora transportation service). This can be made up of an estimated timeuntil the user 110 starts boarding the vehicle 104 (e.g., the timeneeded to walk to the vehicle 104, the time until the vehicle doorsunlock, etc.) and an estimated time of boarding duration for a user 110(e.g., to load luggage, help other passengers board, etc.). In someimplementations, the vehicle computing system 102 can determine anestimated time until the user 110 completes the retrieval of an itemfrom the vehicle 104 (e.g., completely removes an item from the vehicle104 for a delivery service). In some implementations, the vehiclecomputing system 102 can determine an estimated time until the user 110places an item in the vehicle 104 (e.g., for a courier service). Theestimated time estimated time of user arrival (e.g., estimated timeuntil the user 110 completes boarding) can help the vehicle computingsystem 102 determine whether or not to stop at least partially in thetravel way 600. Such time estimate(s) can be expressed as a timeduration (e.g., user estimated to arrive in 1 minute) and/or a point intime (e.g., user estimated to arrive at 10:31 am (PT)).

The estimated time of user arrival (e.g., the estimated time until theuser 110 completes boarding, item retrieval, item placement, etc.) canbe based on a variety of information. FIG. 7 depicts example depicts aflow diagram of an example method 700 of determining an estimated timeof user arrival (e.g., autonomous vehicle user boarding times) accordingto example embodiments of the present disclosure. While the followingprovides examples of the method 700 with respect to determiningautonomous vehicle user boarding times, a similar approach can be takenfor determining an estimated time until the user 110 completes theretrieval of an item from the vehicle 104 and/or an estimated time untilthe user 110 places an item in the vehicle 104. One or more portion(s)of the method 700 can be implemented by one or more computing devicessuch as, for example, the one or more computing device(s) of the vehiclecomputing system 102 and/or other systems (e.g., as computingoperations). Each respective portion of the method 700 can be performedby any (or any combination) of the one or more computing devices.Moreover, one or more portion(s) of the method 700 can be implemented asan algorithm on the hardware components of the device(s) describedherein (e.g., as in FIGS. 1 and 14), for example, to control anautonomous vehicle. FIG. 7 depicts elements performed in a particularorder for purposes of illustration and discussion. Those of ordinaryskill in the art, using the disclosures provided herein, will understandthat the elements of any of the methods discussed herein can be adapted,rearranged, expanded, omitted, combined, and/or modified in various wayswithout deviating from the scope of the present disclosure.

At (702), the method 700 can include obtaining one or more identifiersof a user device. The vehicle computing system 102 can use one or moreidentifier(s) of the user device 138 (e.g., obtained with the request140, obtained by the vehicle 104, provided by the operations computingsystem 106, etc.) to scan for and/or communicate with the user device138 when the vehicle 104 is within the vicinity 206 of the user 110. Byway of example, the operations computing system 106 can provide thevehicle computing system 102 with one or more identifier(s) of a userdevice 138 associated with the user 110. The identifiers can be, forexample, unique radio identifiers (e.g., Bluetooth, WiFi, Cellular,friendly names, contact number, other identifier) collected via asoftware application and provided to the operations computing system106. The vehicle computing system 102 can utilize the identifiers tocommunicate and/or locate a user device 138 associated with the user110. For example, when the vehicle 104 is within a vicinity 206 of thelocation 202 associated with the user 110, the vehicle computing system102 can scan for the user device 138 (e.g., opt-in radios) based atleast in part on the identifier(s) (e.g., when the vehicle 104 is in theapproach mode 108D).

At (704), the method 700 can include obtaining location data associatedwith the user device. For instance, the vehicle computing system 102 canobtain (e.g., via the communication system 136) location data associatedwith a user device 138 associated with a user 110. The location dataassociated with the user device 138 can be indicative of one or morelocations of the user device 138 associated with the user 110, at one ormore times. In some implementations, the vehicle computing system 102can use the identifier(s) to determine the location of the user 110. Thevehicle computing system 102 can be configured to utilize a variety ofcommunication technologies to obtain the location data associated withthe user device 138. For example, the vehicle computing system 102 canobtain the location data based at least in part on a triangulation ofsignals via at least one of multiple input, multiple outputcommunication between the vehicle 104 and the user device 138, one ormore Bluetooth low energy beacons located onboard the vehicle 104, or alight signal handshake between the user device 138 and the vehicle 104,as further described herein.

In some implementations, the vehicle computing system 102 can use radiofrequency (RF) signaling to obtain location data associated with theuser 138. For example, FIG. 8A depicts an example portion 800 of acommunications system 136 according to example embodiments of thepresent disclosure. The communications system 136 can include one ormore electronic devices 802 (e.g., RF module) configured to transmitand/or obtain one or more RF signals (e.g., from the user device 138)via one or more communication device(s) 804 (e.g., transmitters,receivers, transmitters, RF sensors, etc.). For example, once the userdevice 138 is found (e.g., via the identifiers), the vehicle computingsystem 102 can track changes in the signal strength (e.g., radio signalstrength identifier (RSSI)) to determine various information about theuser device 138. For example, the vehicle computing system 102 candetermine the approximate distance of the user device 138 (and the user110) to the vehicle 104 (e.g., the difference between RSSI triangulationand improvements over time can be used to determine the approximatedistance of the user 110 to the vehicle 104). Additionally, oralternatively, the vehicle computing system 102 can determine a headingof the user 110 based at least in part on RF signal(s) associated withthe user device 138 (e.g., the communication device(s) 804 with thebest/strongest RSSI reading can indicate direction/heading of the user110). The vehicle computing system 102 can also determine aspeed/velocity of the user device 138 using such a technique. Theapproximate distance, heading, speed, etc. measurements can bedetermined with or without an authenticated connection.

Additionally, or alternatively, the vehicle computing system 102 canutilize Bluetooth low energy protocol to obtain location data associatedwith the user device 138. For example, FIG. 8B depicts an exampleportion 810 of a communications system 136 according to exampleembodiments of the present disclosure. The communications system 136 caninclude one or more Bluetooth low energy beacons 812 that are locatedonboard the vehicle 104. The vehicle computing system 102 can obtain thelocation data associated with the user device 138 based at least in parton the one or more Bluetooth low energy beacons 812 located onboard thevehicle 104. In some implementations, the vehicle computing system 102can determine differences in Bluetooth Low Energy (BLE) beacon radiosignal strength identifiers over time and/or inertial measurement unitchanges, which can indicate a distance between the user device 138 ofthe user 110 (e.g., a mobile phone associated with the user 110) and thevehicle 104. Additionally, or alternatively, the vehicle computingsystem 102 can determine a heading of the user device 138 (and the user110) based at least in part on the Bluetooth low energy beacon(s) 812(e.g., the beacon(s) with the best/strongest RSSI reading can indicatedirection/heading of the user 110). The vehicle computing system 102 canalso determine a speed/velocity of the user device 138 using such atechnique. In some implementations, the user device 138 can determineits location (e.g., relative to the vehicle 104) based at least in parton signals transmitted from one or more beacons 812 located onboard thevehicle 104. The user device 138 can provide data indicative of itslocation to one or more remote computing device(s) (e.g., the operationscomputing system 106, cloud-based system). The remote computingdevice(s) can provide data indicative of the location of the user device138 (determined based on the beacons 812) to the vehicle computingsystem 102. The vehicle computing system 102 can determine the locationof the user device 138 (and the user 110) based on such data. In someimplementations, the remote computing device(s) can process data fromthe user device 138 (e.g., associated with the beacons 812) to determinea location of the user device 138 and provide data associated therewithto the vehicle computing system 102.

In some implementations, the vehicle computing system 102 can utilizeone or more altimeters (and/or other measuring device) to obtainlocation data associated with the user device 138. For example, FIG. 8Cdepicts an example diagram 820 of obtaining location data according toexample embodiments of the present disclosure. The vehicle 104 caninclude one or more altimeters 822 located onboard the vehicle 104. Theuser device 138 can include one or more altimeters 824. As shown, theuser device 138 (and the user 110) can be located on the second story ofa building 826. The vehicle computing system 102 can obtain locationdata associated with the user device 138 via at least one altimeter 822located onboard the vehicle 104. By way of example, the vehiclecomputing system 102 can obtain location data associated withaltimeter(s) 824 of the user device 138 via one or more network(s) 828.The vehicle computing system 102 can compare the location dataassociated with altimeter(s) 824 of the user device 138 to location dataassociated with altimeter(s) 822 of the vehicle 104. The vehiclecomputing system 102 can determine an elevation/altitude of the userdevice 138 (and the user 110) based at least in part on this comparison(e.g., a difference between the altimeter(s) 824 of the user device 138and the altimeter(s) 822 of the vehicle 104 can indicateelevation/altitude difference). The elevation/altitude can be relativeto the position of the vehicle 104 and/or another reference (e.g.,ground level, sea level, etc.).

In some implementations, the vehicle computing system 102 can utilizeother communication technologies to obtain location data associated withthe user device 138. For example, the vehicle computing system 102 canobtain the location data associated with the user device 138 based atleast in part on multiple input, multiple output communication betweenthe vehicle 104 and the user device 138 associated with the user 110.This can allow the vehicle 104 to take advantage of the multipleantennas included in the vehicle's communication system 136 as well asthose of the user device 138 to increase accuracy of the location data704A associated with the user 110. For example, the user device 138 canobtain an identifier (e.g., Radio Network Temporary Identifier) that canbe associated with the user device 138. The user device 138 can providedata indicative of the identifier to one or more remote computingdevice(s) (e.g., the operations computing system 106, cloud-basedsystem). The remote computing device(s) can provide data indicative ofthe identifier to the vehicle computing system 102. The vehiclecomputing system 102 can locate the user device 138 (and the user 110)based at least in part on the identifier via the multiple antennasonboard the vehicle 104 (and/or the antenna(s) of the user device 138).In some implementations, the vehicle computing system 102 can utilize ahandshake (e.g., light signal handshake) between the user device 138 andthe vehicle 104.

In some implementations, the vehicle computing system 102 can obtainlocation data associated with the user device 138 based at least in parton image data. For example, the user 110 can obtain image dataassociated with the user 110 (e.g., via the user device 138). The imagedata can be indicative of one or more characteristics (e.g., buildings,street signs, etc.) of the geographic area and/or surroundingenvironment of the user 110. In some implementations, the user 110 maybe included in the image data. The user device 138 can process the imagedata to determine the location of the user 110 (e.g., via a comparisonof image data features to known geographic features). The user device138 can provide the determined location to one or more remote computingdevice(s) (e.g., the operations computing system 106 and/or othercloud-based system). The remote computing device(s) can provide dataindicative of the location of the user 110 to the vehicle computingsystem 102. The vehicle computing system 102 can obtain the dataindicative of the location of the user 110 determined based at least inpart on image data associated with the user 110. The estimated timeuntil the user 110 starts boarding the vehicle 104 can be based at leastin part on the location of the user 110. In some implementations, theremote computing device (e.g., the operations computing system 106and/or other cloud-based system) can obtain the image data associatedwith the user 110, process the image data (e.g., as described herein),and provide the data indicative of the location of the user 110 to thevehicle computing system 102. This can be helpful to save the processingresources of a computationally limited device (e.g., mobile device).

In some implementations, the vehicle computing system 102 can obtain theimage data associated with the user 110 (e.g., via the user device 138and/or the remote computing device). The vehicle computing system 102can determine a location of the user 110 based at least in part on theimage data. For example, the vehicle computing system 102 can analyzethe features of the image data (e.g., the background, street signs,buildings, etc.) and compare the features to other data (e.g., sensordata 118, map data 120, other data, etc.). The vehicle computing system102 can determine a location of the user based at least in part on thiscomparison.

In some implementations, the vehicle computing system 102 can utilizeother communication techniques. These techniques can include, forexample, vehicle perception of the user 110 (e.g., via processing ofsensor data 118 to perceive the user 110 and the user's location,distance, heading, velocity, and/or other state data 130 associatedtherewith), GPS location of the user device 138, device specifictechniques (e.g., specific device/model type), the vehicle 104 servingas a localized base station (e.g., GPS, WiFi, etc.), and/or othertechniques.

Returning to FIG. 7, the method 700 can include determining an estimatedtime until the user starts interaction with the vehicle, at (706). Forinstance, the vehicle computing system 102 can determine an estimatedtime until the user starts boarding the vehicle 104 (ETSB) based atleast in part on the location data associated with the user device 138.The estimated time of until the user 110 starts interaction with thevehicle 104 (e.g., the estimated time until the user starts boarding thevehicle 104 (ETSB)) can be indicative of, for example, a countdown intime until the user's location is near/at the vehicle 104, until theuser 110 begins to board, until the doors of the vehicle 104 areunlocked, until the doors are opened, etc. The user 110 can, but neednot, physically interact with the vehicle 104 for this time estimate.

The estimated time until the user 110 starts interaction with thevehicle 104 (e.g., the estimated time until the user starts boarding thevehicle 104 (ETSB)) can be based on a variety of data. For example, thevehicle computing system 102 can determine a distance 708 between theuser 110 and the vehicle 104 based at least in part on the location dataassociated with the user device 138, as described herein. The estimatedtime until the user 110 starts interaction with the vehicle 104 (e.g.,the estimated time until the user starts boarding the vehicle 104(ETSB)) can be based at least in part on the distance 708 between theuser 110 and the vehicle 104. Additionally, or alternatively, thevehicle computing system 102 can determine an elevation/altitude 710 ofthe user 110 based at least in part on the location data associated withthe user device 138 (e.g., obtained via the one or more altimetersonboard the vehicle 104), as described herein. The estimated time untilthe user 110 starts interaction with the vehicle 104 (e.g., theestimated time until the user starts boarding the vehicle 104 (ETSB))can be based at least in part on the elevation/altitude 710 of the user110 (e.g., if the user 110 is in a tall building then additional timeshould be added). Additionally, or alternatively, the vehicle computingsystem 102 can determine a heading 712 of the user 110 based at least inpart on the location data associated with the user device 138, asdescribed herein. The estimated time until the user 110 startsinteraction with the vehicle 104 (e.g., the estimated time until theuser starts boarding the vehicle 104 (ETSB)) can be based at least inpart on the heading 712 of the user 110 (e.g., if the user 110 is acrossthe street then additional time can be added). Additionally, oralternatively, the vehicle computing system 102 can determine a locationof the user 110 based at least in part on image data, as describedherein. The estimated until the user 110 starts interaction with thevehicle 104 (e.g., the estimated time until the user starts boarding thevehicle 104 (ETSB)) can be based at least in part on the location of theuser 100, as determined from the image data.

In some implementations, the vehicle computing system 102 can obtainhistoric data 714 to help determine the estimated time until the user110 starts interaction with the vehicle 104 (e.g., the estimated timeuntil the user starts boarding the vehicle 104 (ETSB)). The historicdata 714 can be indicative of, for example, historic start boardingtimes of one or more other users (and or the user 110). In someimplementations, a machine learned model can be trained based on suchhistoric data to determine the estimated time until the user 110 startsinteraction with the vehicle 104 (e.g., the estimated time until theuser starts boarding the vehicle 104 (ETSB)). For example, the model canbe trained based on training data indicative of previous location data,user distances, altitudes, headings, etc. labeled with the times of userarrival (e.g., the times when the users started boarding the vehicle).The model can be trained to receive input data (e.g., location data,user distances, altitudes, headings, etc.) and provide, as an output, anuntil the user 110 starts interaction with the vehicle 104 (e.g., anestimated time until the user starts boarding a vehicle (ETSB)). In someimplementations, the estimated time until the user 110 startsinteraction with the vehicle 104 (e.g., the estimated time until theuser starts boarding the vehicle 104 (ETSB)) can be based at least inpart on the historic data 714. For example, vehicle computing system 102can determine the estimated time until the user 110 starts interactionwith the vehicle 104 (e.g., the estimated time until the user startsboarding the vehicle 104 (ETSB)) based at least in part on such amachine-learned model (e.g., as an output thereto).

In some implementations, the estimated time until the user 110 startsinteraction with the vehicle 104 (e.g., the estimated time until theuser starts boarding the vehicle 104 (ETSB)) can be used to determineone or more vehicle actions, at (716). For example, the estimated timeuntil the user 110 starts interaction with the vehicle 104 (e.g., theestimated time until the user starts boarding the vehicle 104 (ETSB))can indicate that the user 110 is close to the vehicle 104 and/orheading toward the vehicle 104. The vehicle computing system 102 cancause one or more doors of the vehicle 102 to unlock based at least inpart on the estimated time until the user 110 starts interaction withthe vehicle 104 (e.g., the estimated time until the user starts boardingthe vehicle 104 (ETSB)). To do so, the vehicle computing system 102 canprovide control signal(s) to an associated door controller.Additionally, or alternatively, the vehicle computing system 102 cancause the vehicle 104 to implement one or more vehicle settings (e.g.,temperature, music, etc.) associated with the user 110 based at least inpart on the estimated time until the user 110 starts interaction withthe vehicle 104 (e.g., the estimated time until the user starts boardingthe vehicle 104 (ETSB)). For example, the vehicle computing system 102can access a profile associated with the user 110 to identity the user'spreferred vehicle settings and can provide one or more control signalsto the appropriate systems onboard the vehicle 104 (e.g., temperaturecontrol system, sound system etc.) to implement the vehicle settings.

At (718), the method 700 can include determining an estimated time ofinteraction duration between the user and the vehicle. This estimatedtime can be indicative of the time it will take for the user 110 tointeract with the vehicle 102. By way of example, the estimated time ofinteraction duration can include an estimated board duration (EBD) thatis indicative of how long the user 110 may take to load him/herself,children, luggage, etc. into the vehicle 104, to securely fasteningseatbelts, and/or undertake other tasks (e.g., for a transportationservice). In some implementations, the estimated time of interactionduration can be indicative of how long the user 110 may take to unsecureand remove an item from the vehicle 104 (e.g., for a delivery service).In some implementations, the estimated time of interaction duration canbe indicative of how long the user 110 may take to securely place anitem into the vehicle 104 (e.g., for a courier service).

To help determine the estimated time of interaction duration between theuser 110 and the vehicle 104, the vehicle computing system 102 canobtain data associated with the user 110. The vehicle computing system102 can determine the estimated time of interaction duration (e.g., anestimated time of boarding duration for the user 110) based at least inpart on the data associated with the user 110. The data associated withthe user 110 can include, for example, data indicative of one or morepreferences 720 of the user 110 and/or one or more vehicle serviceparameters 722 associated with the user 110. The preferences 720 can beindicative of the user's destination (e.g., airport, train station,etc.), service type, and/or other information specified by the user 110(e.g., when requesting the vehicle service). The one or more vehicleservice parameters 722 can be indicative of number of passengers,child's car seat request, presence/amount of luggage, etc. The vehicleservice parameters 722 can also be specified by the user 110 (e.g., whenrequesting the vehicle service).

In some implementations, the data associated with the user 110 caninclude historic data 724 (e.g., indicative of a boarding behaviorassociated with one or more other users). For example, the historic data724 can be indicative of historic wait time(s) associated with otherusers in the geographic area 200 and/or a greater region, similarlysituated users, etc. In some implementations, the historic data 724 canbe associated with the specific user 110. The historic data 724 caninclude, for example, previous correlations between changes in thesignal strength of an identifier and a user's time to arriving at avehicle. For example, the historic data 724 can indicate historic RSSIchanges as a countdown to user arrival.

At (726), the method 700 can include determining an estimated time ofuser arrival with the vehicle 104. The estimated time of user arrivalcan include, for example, an estimated time until the user completesboarding of the vehicle 104 (ETCB) (e.g., for a transportation service),an estimated time until the user 110 finishes retrieving an item fromthe vehicle 104 (e.g., for a delivery service), and estimated time untilthe user 110 finishes placing an item in the vehicle 104 (e.g., for acourier service), etc. The vehicle computing system 102 can determine anestimated time of user arrival based at least in part on the locationdata associated with the user device 138. The vehicle computing system102 can determine the estimated time of user arrival based at least inat part on the estimated time until the user 110 starts interaction withthe vehicle 104 and the estimated time of interaction duration (e.g., asum of these estimated times). By way of example, the vehicle computingsystem 102 can determine an estimated time until the user 110 completesboarding of the vehicle 102 based at least in part on the location dataassociated with the user device 110 and the data associated with theuser 110. More particularly, the vehicle computing system 102 candetermine an estimated time until the user 110 completes boarding of thevehicle 104 based at least in part on the estimated time until the user110 starts boarding the vehicle 104 and the estimated time of boardingduration for the user 110. The vehicle computing system 102 candetermine the estimated time until the user 110 completes boarding ofthe vehicle 104 via the addition of these two time estimates (e.g.,ETCB=ETSB+EBD). In this way, the vehicle computing system 102 candetermine the amount of time that the object(s) within its surroundingswould be impacted as the vehicle 104 waits for the user 110 (e.g., howlong other vehicles would be caused to stop while waiting for the user110 to board the vehicle 104).

In some implementations, in order to determine whether the estimatedtime of user arrival (e.g., the estimated time until the user 110completes boarding of the vehicle 104) is acceptable, the vehiclecomputing system 102 can compare the estimated time for interactionbetween the user 110 and the vehicle 104 (e.g., the estimated time untilthe user 110 completes boarding of the vehicle 104) to a time constraint750. The time constraint 750 can be expressed as a time threshold (e.g.,indicating an acceptable amount of stopping time) and/or cost data(e.g., cost functions expressing a cost in relation to stopping time).This can allow the vehicle computing system 102 to determine whether theamount of stopping time is acceptable.

Similar to the traffic constraint 506, the time constraint 750 can bebased on historic data (e.g., indicating historic wait times), real-timedata (e.g., indicating that the vehicles are already waiting due toanother traffic build-up in front of the autonomous vehicle),expectations of individuals in the geographic area, machine-learnedmodel(s), and/or other information. For example, in the event that thereis already a traffic jam in front of the vehicle 104, the timeconstraint 750 (e.g., indicative an acceptable wait time) can be higher.

In some implementations, the time constraint 750 can be determined atleast in part from a model, such as a machine-learned model. Forexample, the machine-learned model can be or can otherwise include oneor more various model(s) such as, for example, models using boostedrandom forest techniques, neural networks (e.g., deep neural networks),or other multi-layer non-linear models. Neural networks can includerecurrent neural networks (e.g., long short-term memory recurrent neuralnetworks), feed-forward neural networks, and/or other forms of neuralnetworks. For instance, supervised training techniques can be performedto train the model (e.g., using historical wait time data) to determinea time constraint 750 based at least in part on input data. The inputdata can include, for example, the data described herein with referenceto FIGS. 7 and 8A-C, driving characteristics of individuals in anassociated geographic area, complaints received from operators ofvehicles that were caused to stop by autonomous vehicles, etc. Themachine-learned model can provide, as an output, data indicative of arecommended time constraint. In some implementations, the recommendedtime constraint can be specific to the geographic area 200. For example,the model can be trained based at least in part on data associated withgeographic area 200 such that the recommended time constraint isspecific for that particular region.

Additionally, or alternatively, one or more time estimates of FIG. 7 canbe based at least in part on a model. The vehicle computing system 102can obtain data descriptive of the model (e.g., machine learned model).The vehicle computing system 102 can provide input data to the model.The input data can include the one or more of the types of datadescribed herein with reference to FIGS. 7 and 8A-C. For example, themodel can determine an estimated time of user arrival (e.g., anestimated time until the user 110 completes boarding of the vehicle 104(ETCB)), an estimated time until the user 110 starts interaction withthe vehicle 104 (e.g., an estimate time until the user 110 startsboarding the vehicle 104 (ETSB)), and/or an estimated time ofinteraction duration between the user 110 and the vehicle 104 (e.g., anestimated time of boarding duration for a user 110 (EBD)). For example,the estimated time of user arrival to the vehicle 104 (e.g., ETSB) canbe based at least in part on the input data (e.g., the user's location,heading, distance, altitude, etc.). In some implementations, the modelcan analyze the input data with respect to the time constraint 750 todetermine whether the estimated time of user arrival (e.g., theestimated time until the user 110 completes boarding of the vehicle 104(ETCB)) is high or low, significant or insignificant, acceptable orunacceptable, etc. The output of such a model can be the estimated timefor interaction between the user 110 and the vehicle 104 (e.g., theestimated time until the user 110 completes boarding of the vehicle 104(ETCB)) and/or whether it is high or low, significant or insignificant,acceptable or unacceptable, etc.

In some implementations, the output of the model can be provided as aninput to the model for another set of data (e.g., at a subsequent timestep). In such fashion, confidence can be built that a determined timeestimate is the accurate. Stated differently, in some implementations,the process can be iterative such that the time estimate can berecalculated over time as it becomes clearer what the time estimate iswith respect to the user 110. For example, the model can include one ormore autoregressive models. In some implementations, the model caninclude one or more machine-learned recurrent neural networks. Forexample, recurrent neural networks can include long or short-term memoryrecurrent neural networks, gated recurrent unit networks, or other formsof recurrent neural networks.

Returning to FIG. 6, the vehicle computing system 102 can determine oneor more vehicle actions based at least in part on at least one of theestimated traffic impact 504 or the estimated time of user arrival(e.g., one and/or both of the estimates). In some implementations, thevehicle computing system 102 can determine the vehicle action(s) basedat least in part on the estimated traffic impact 504. By way of example,the vehicle action(s) can include stopping within the vicinity 206 ofthe location 202 associated with the user 110 (e.g., at least partiallyin the travel way 600). In the event that the level of traffic (e.g.,the number of other vehicles that would be impacted by an in-lane stop)is below a traffic threshold (e.g., the estimated traffic impact 504 islow), the vehicle computing system 102 can determine that the vehicle104 can stop within the travel way 600 to wait for the user 110 toarrive at the vehicle 104. In some implementations, the vehiclecomputing system 102 can determine the vehicle action(s) based at leastin part on the estimated time of user arrival. For instance, in theevent that the estimated time of user 702 arrival is below the timeconstraint 706 (e.g., the estimated time of user arrival is low), thevehicle computing system 102 can determine that the vehicle 104 can stopat least partially within the travel way 600. Such a stop can occur, forexample, as close as possible (e.g., for the vehicle 104) to thelocation 202 associated with the user 110.

In some implementations, the vehicle computing system 102 can base itsdetermination to stop at least partially within the travel way 600 onboth the estimated traffic impact 504 and the estimated time of userarrival. For instance, vehicle computing system 102 can weigh each ofthese estimates to determine whether it would be appropriate for thevehicle 104 to stop at least partially in the travel way 600 to wait forthe user 110. The vehicle computing system 102 can apply a firstweighting factor to the estimated traffic impact 504 and a secondweighting factor to the estimated time of user arrival. The firstweighting factor can be different than the second weighting factor. Insome implementations, the first weighting factor can be inverselyrelated to the second weighting factor. An example equation can include:ETI*w1+ETUR*w2, where “ETI” is the estimated traffic impact 504, “w1” isthe first weighing factor (e.g., 0 to 1), “ETUR” is the estimated timeto user arrival 702, and “w2” is the second weighting factor (e.g., 0 to1). By way of example, the estimated traffic impact 504 may be highwhile the estimated time of user arrival may be short. Accordingly, thevehicle computing system 102 may determine that it can stop within thetravel way 600 because although a higher number of objects (e.g., othervehicles) may be caused to stop, it would only be for a short period oftime because the user 110 is close in distance to and/or quickly headingtoward the vehicle 104. In such a case, the estimated time of userarrival can be given a greater weight than the estimated traffic impact504. In another example, the estimated impact on traffic 504 may be lowwhile the estimated time of user arrival may be long. Accordingly, thevehicle computing system 102 can determine that it should not stopwithin the travel way 600 because although only a few objects (e.g.,other vehicles) may be caused to stop, it would be for a greater periodof time because the user 110 is farther from (and/or moving slowly,moving away from, etc.) the vehicle 104. In such a case, the estimatedtime of user arrival can be given a greater weight than the estimatedtraffic impact 504. In some implementations, the first and secondweighting factors can be manually and/or automatically adjusteddepending on the circumstances (e.g., a VIP user is being provided thevehicle services) and/or the geographic area 200.

In some implementations, the estimated traffic impact 504 can beadjusted based at least in part on the estimated time to user arrival702. For instance, in the event that the estimated time to user arrival702 is long, the estimated traffic impact 504 can be higher.

With reference again to FIG. 2, the vehicle computing system 102 canalso, or alternatively determine that the vehicle 104 is to enter into aholding pattern. For instance, the vehicle action(s) can includetraveling along a second vehicle route 208 (e.g., an optimal holdingpattern route). In some implementations, the vehicle computing system102 can cause the vehicle 104 to enter into a particular operating modein which the vehicle 104 implements a holding pattern (e.g., a holdingpattern operating mode). The vehicle 104 may be unable to find a parkinglocation before and/or after travelling passed the location 202associated with the user 110. Additionally, the vehicle 104 maydetermine that it should not stop within a travel way 600 to wait forthe user's arrival, as described herein. Thus, the vehicle 104 can bere-routed along a second vehicle route 208 that is at least partiallydifferent than the first vehicle route. The second vehicle route 208 canbe a path along which the vehicle 104 can travel to re-arrive within thevicinity 206 of the location 202 of the user 110. For example, thesecond vehicle route 208 can be a path along which the vehicle 104 cantravel around a block, back to the location associated with the user. Insome implementations, such a path may be similar to and/or the same as aportion of the first vehicle route 204 (e.g., along the streetassociated with the user 110). In some implementations, the secondvehicle route 208 can be completely different from the first vehicleroute 204 such that no portion of the second vehicle route 208 overlapswith the first vehicle route 204.

The determination of the second vehicle route 208 can be based on avariety of information. For example, FIG. 9 depicts example information900 associated with a second vehicle route 208 according to exampleembodiments of the present disclosure. In some implementations, thevehicle computing system 102 can determine the second vehicle route 208based at least in part on such information. In some implementations, thesecond vehicle route 208 can be determined off-board the vehicle 104 byanother computing system (e.g., the operations computing system 106) anddata indicative of the second vehicle route 208 can be provided to thevehicle computing system 102.

The second vehicle route 208 can be determined based at least in part oncurrent and/or historic traffic data 902A. For example, the secondvehicle route 208 can be determined to implement the route that willallow the vehicle 104 to arrive back within the vicinity 206 of thelocation 202 of the user 110 within the shortest amount of time and/ordistance. The second vehicle route 208 can take into account the currenttraffic (and/or historic traffic patterns) within the geographic area200 such that the vehicle 104 is minimally impeded by such traffic(e.g., such that the second vehicle route 208 is the fastest and/orshortest vehicle route to the location 202 of the user 110).

Additionally, or alternatively, the second vehicle route 208 can bedetermined based at least in part on map data 902B. For example, the mapdata 902B can be used to determine the path (e.g., roads, other terrain,etc.) the vehicle 104 is to travel along to arrive back within thevicinity 206 of the user 110.

In some implementations, the second vehicle route 208 can be based ondata 902C associated with other vehicle(s) in the geographic area 200.For instance, the data 902C associated with other vehicle(s) can includeadditional traffic data associated with the geographic area 200 (e.g.,indicating a certain road is impeded by heavy traffic). The data 902Cassociated with the other vehicle(s) can also include the location ofsuch vehicles. For example, the vehicle computing system 102 and/or theoperations computing system 106 can determine the second vehicle route208 (e.g., optimal vehicle holding pattern) by processing map data andtraffic data to establish an estimated time back to the location 202associated with the user 110. If another vehicle (e.g., anotherautonomous vehicle in an associated fleet) can arrive at the location202 associated with the user 110 in a shorter time period than thevehicle 104, the other vehicle (rather than the vehicle 104) can berouted to the location 202. If the vehicle 104 would arrive to thelocation 202 the fastest, then the vehicle 104 can be routed inaccordance with the second vehicle route 208. In some implementations,the second vehicle route 208 can be based on a model, such as amachine-learned model, in a manner similar to that described herein withrespect to the estimated traffic impact 504, the estimated time to userarrival 702, etc.

In some implementations, the vehicle computing system 102 can determinethat the vehicle 104 can stop within the travel way 600, but laterdetermine that the vehicle 104 should begin to travel again (e.g.,according to a holding pattern route). For example, the vehiclecomputing system 102 can determine that it would be appropriate for thevehicle 104 to stop at least partially within the travel way 600 to waitfor the user 110 based at least in part on the estimated traffic impact504 and/or the estimated time of user arrival, as described herein. Thevehicle computing system 102 can be configured to update (e.g.,continuously, periodically, as scheduled, in real-time, in nearreal-time, etc.) the estimated traffic impact 504 and/or the estimatedtime of user arrival. For example, while the vehicle 104 is stopped, thetraffic impact may increase (e.g., due to an increase in the number ofother vehicle(s) stopped behind the vehicle 104) and/or the user 110 maytake longer than estimated to arrive at the vehicle 104. The vehiclecomputing system 102 can determine at least one of an updated estimatedtraffic impact (e.g., based on the number of vehicles that have alreadystopped and/or additional vehicles that may be caused to stop) or anupdated estimated time of user arrival (e.g., based on a change in theuser device location data, if any). The vehicle computing system 102 candetermine that the vehicle 104 can no longer remain stopped to wait forthe user 110 based at least in part on at least one of the updatedestimated traffic impact or the updated estimated time of user arrival.Accordingly, the vehicle computing system 102 can cause the vehicle 104to travel along the second vehicle route 208 based at least in part onat least one of the updated estimated traffic impact or the updatedestimated time of user arrival.

The vehicle computing system 102 can cause the vehicle 104 to performone or more vehicle action(s). As described herein, the vehicleaction(s) can include at least one of stopping the vehicle 104 (e.g., atleast partially within the travel way 600) within the vicinity 206 ofthe location 202 associated with the user 110 or travelling along asecond vehicle route 208. In the event that the one or more vehicleactions include stopping the vehicle 104 within the vicinity 206 of thelocation 202 associated with the user 110, the vehicle computing system102 can cause the vehicle 104 to stop. For example, the vehiclecomputing system 102 can provide one or more control signals to acontrol system 116 of the vehicle 104 (e.g., braking control system) tocause the vehicle 104 to decelerate to a stopped position that is atleast partially in a travel way 600 within the vicinity 206 of thelocation 202 associated with the user 110. In the event that the vehiclecomputing system 102 has determined that the vehicle 104 is to travelalong a second vehicle route 208 (e.g., in accordance with the holdingpattern), the vehicle computing system 102 can obtain data associatedwith the second vehicle route 208 and implement the second vehicle route208 accordingly. For example, the vehicle computing system 102 canrequest and obtain data indicative of the second vehicle route 208 fromthe operations computing system 106. Additionally, or alternatively, thevehicle computing system 102 can determine the second vehicle route 208onboard the vehicle 104. The vehicle computing system 102 can provideone or more control signals to cause the vehicle 104 to implement amotion plan that causes the vehicle 104 to travel in accordance with thesecond vehicle route 208 (e.g., to implement one or more vehicletrajectories in accordance with the second vehicle route 208).

The vehicle computing system 102 can provide the user 110 with one ormore communications indicating the actions performed by (or to beperformed by) the vehicle 104. For example, in the event that thevehicle 104 does not find a parking location and does not stop in thetravel way, the vehicle computing system 102 can provide, via thecommunication system 136, a communication to the user device 138associated with the user 110. The communication can include, forexample, a textual message, auditory message, etc. that indicates thevehicle actions (e.g., “I could not locate you at the pin drop, trafficforced me to go around the block. Please proceed to the pin drop”).

In the event that the vehicle 104 stops at least partially within thetravel way 600, the vehicle computing system 102 can provide acommunication (e.g., data) to a user device 138 associated with the user110. The communication can indicate that the vehicle 104 is travellingto return to the location 202 associated with the user 110. In responseto receiving such a communication, the user device 138 associated withthe user 110 can display a user interface indicative of thecommunication. For example, FIG. 10 depicts an example display device1000 with an example communication 1002 according to example embodimentsof the present disclosure. The display device 1000 (e.g., displayscreen) can be associated with the user device 138 associated with theuser 110. The communication 1002 can be presented via a user interface1004 on the display device 1000. The communication 1002 can indicatethat the vehicle 104 has arrived and is waiting in the travel way (e.g.,in a current traffic lane). In some implementations, the communication1002 and/or another portion of the user interface 1004 can be indicativeof a location of the vehicle 104. For example, the display device 1000can display a map user interface 1006 that includes a user route 1008.The user route 1008 can be a route along which a user 110 can travel toarrive at the vehicle 104.

Returning to FIG. 2, in some implementations, the vehicle 104 may berelieved of its responsibility to provide a vehicle service to the user110. For instance, in the event that a vehicle computing system 102(and/or operations computing system 106) determines that a vehicle 104is to travel along the second vehicle route 208, such computingsystem(s) can determine whether it would be advantageous (e.g., moretime efficient, more fuel efficient, etc.) for another vehicle 210(e.g., another autonomous vehicle) within the geographic area 200 to berouted to the user 110. In the event that it would be advantageous(e.g., because the other vehicle 210 can arrive sooner), the operationscomputing system 106 can provide data to the vehicle 104 indicating thatthe vehicle 104 is no longer responsible for the request 140. The othervehicle 210 can be routed to the user 110 in the manner describedherein. Additionally, or alternatively, the operations computing system106 (and/or the vehicle computing system 102 of the vehicle 104) canre-route the vehicle 104 to provide a vehicle service to another user.

In some implementations, the vehicle computing system 102 can cancel therequest 140 associated with the user 110. By way of example, the vehicle104 may be caused to re-route (e.g., circle the block) a certain numberof times and/or the user 110 may not arrive at the vehicle 104 within acertain timeframe. The vehicle computing system 102 can determinewhether to cancel the request 140 based at least in part on a vehicleservice cancellation threshold.

FIG. 11 depicts example information 1100 associated with a vehicleservice cancellation threshold 1102 according to example embodiments ofthe present disclosure. The vehicle service cancellation threshold 1102can be indicative of a threshold time and/or distance that the vehicle104 is in the holding pattern. For instance, the vehicle servicecancellation threshold 1102 can be indicative of a time between when thevehicle 104 initially arrived within a vicinity 206 of the user 110(and/or passed the location 202) to the current time. Additionally, oralternatively, the vehicle service cancellation threshold 1102 can beindicative of a distance traveled by the vehicle 104 while in a holdingpattern (e.g., number of times the vehicle is re-routed to arrive at theuser 110). The vehicle service cancellation threshold 1102 can bedetermined and updated continuously, periodically, as scheduled, onrequest, in real-time, in near real-time, etc. (e.g., per trip, while ona trip, etc.). The vehicle service cancellation threshold 1102 can bedetermined by the vehicle computing system 102 and/or off-board thevehicle 104 and provided to the vehicle computing system 102.

The information 1100 can include vehicle service demand data 1104A,historic vehicle service data 1104B, geographic area preferences 1104C,data 1104D associated with other vehicle(s) within the geographic area,user selected holding patterns 1104E, and/or other types of information.The vehicle service demand data 1104A can include a current level ofdemand (e.g., number of current service requests) for vehicle services(e.g., within the geographic area 200). Additionally, or alternatively,the vehicle service demand data 1104A can include a historic level ofdemand for vehicle services at a certain time, day, etc. (e.g., withinthe geographic area, similarly situated area, etc.). In the event thatthe demand is lower, the vehicle service cancellation threshold 1102 canbe higher (e.g., because the vehicle 104 may not be needed for othervehicle service requests). In the event that the demand is higher, thevehicle service cancellation threshold 1102 can be lower (e.g., becausethe vehicle 104 may be needed for other vehicle service requests).

The historic vehicle service data 1104B can be associated with thespecific user 110 and/or one or more other user(s). For example, thehistoric vehicle service data 1104B can indicate that the user 110typically takes a longer amount of time to arrive at the vehicle 104(e.g., due to a disability). As such, the vehicle service cancellationthreshold 1102 can be higher in order to cause the vehicle 104 to remainin the holding pattern longer, thereby giving the user 110 a greateropportunity to arrive at the vehicle 104. Additionally, oralternatively, the historic vehicle service data 1104B can indicate thatit generally takes user(s) of within the geographic area 200 longer toarrive at the vehicle 104. As such, the vehicle service cancellationthreshold 1102 may be higher. In the event that the historic vehicleservice data 1104B indicates that the user 110 (and/or other user(s))typically arrives at the vehicle 104 in a relatively short timeframe,the vehicle service cancellation threshold 1102 may be lower.

The geographic area preferences 1104C can be descriptive of thepreferences associated with a geographic area 200 (e.g., as indicated bythe managers of the geographic area 200). For example, the geographicarea 200 may prefer that a vehicle 104 only remain in a holding pattern(e.g., circle the block) for a certain time period and/or distance so asnot to affect local traffic.

In some implementations, the vehicle service cancellation threshold 1102can be based at least in part on data 1104D associated with one or moreother vehicles (e.g., other autonomous vehicles in an associated fleet).For example, the data 1104D can be indicative of the location(s) ofother vehicle(s) (e.g., within the geographic area 200) and/or whetherthe other vehicle(s) are available to provide a vehicle service (e.g.,whether or not the other vehicle is assigned to a service request,currently providing a vehicle service, etc.). In the event that anothervehicle is located close to the location 202 associated with the user110, the vehicle service cancellation threshold 1102 may be lower (e.g.,because the user 110 can be serviced by the other vehicle in the event anew request is made after cancellation). In the event that anothervehicle is not located close to the location 202 associated with theuser 110, the vehicle service cancellation threshold 1102 may be higher(e.g., because another vehicle is not readily available in the event theuser 110 makes a new request after cancellation).

In some implementations, the vehicle service cancellation threshold 1102can be based at least in part on user selected holding patterns 1104E.The user selected holding patterns 1104E can include data indicative ofa vehicle service cancellation threshold 1102 selected by a user 110.For example, the user 110 can purchase (e.g., via a user interfaceassociated with a software application) a higher vehicle servicecancellation threshold 1102, such that the vehicle 104 will remain inthe holding pattern (e.g., circle the block) for a longer time/distancebefore the vehicle service request is cancelled. In someimplementations, a user 110 can have a higher vehicle servicecancellation threshold 1102 due to a higher user rating, specializedtreatment (e.g., frequent user), and/or based on other conditions.

In some implementations, the vehicle service cancellation threshold 1102can be based at least in part on a model, such as a machine learnedmodel. For example, the model can be trained based on previouslyobtained information 1100 and labeled data indicative of the vehicleservice cancellation thresholds associated therewith. In a mannersimilar to that described herein for the estimated traffic impact 504and the estimated time of user arrival, the vehicle computing system 102(or other computing system) can provide input data (e.g., theinformation 1100) into such a model and receive, as an output, arecommended vehicle service cancellation threshold.

The vehicle computing system 102 can cancel the request 140 associatedwith the user 110 in the event that the user 110 has not arrived at thevehicle 104 and the vehicle 104 has exceeded the vehicle servicecancellation threshold 1102. In response, the vehicle computing system102 can provide data indicating that the request 140 for the vehicleservice provided by the vehicle 104 is cancelled to the operationscomputing system 106 (and/or one or more other computing devices thatare remote from the vehicle computing system 102). In someimplementations, such data can request the cancellation of the user'sservice request 140. The operations computing system 106 can cancel theservice request 140 (and inform the user 110 accordingly) and/orre-route the vehicle 104 to provide a vehicle service to another user.In some implementations, the vehicle computing system 102 cancommunicate directly with a user device 138 associated with the user 110to cancel the service request 140 and/or inform the user 110 of thevehicle service cancellation. The vehicle computing system 102 canreport such a cancellation to the operations computing system 106.

FIG. 12 depicts a flow diagram of an example method 1200 of controllingautonomous vehicles according to example embodiments of the presentdisclosure. One or more portion(s) of the method 1200 can be implementedby one or more computing devices such as, for example, the one or morecomputing device(s) of the vehicle computing system 102 and/or othersystems. Each respective portion of the method 1200 (e.g., 1202-1222)can be performed by any (or any combination) of the one or morecomputing devices. Moreover, one or more portion(s) of the method 1200can be implemented as an algorithm on the hardware components of thedevice(s) described herein (e.g., as in FIGS. 1 and 14), for example, tocontrol an autonomous vehicle. FIG. 12 depicts elements performed in aparticular order for purposes of illustration and discussion. Those ofordinary skill in the art, using the disclosures provided herein, willunderstand that the elements of any of the methods discussed herein canbe adapted, rearranged, expanded, omitted, combined, and/or modified invarious ways without deviating from the scope of the present disclosure.

At (1202), the method 1200 can include obtaining data indicative of alocation associated with a user. For instance, the vehicle computingsystem 102 can obtain data 142 indicative of a location 202 associatedwith a user 110 to which the vehicle 104 is to travel. The vehicle 104is to travel along a first vehicle route 204 that leads to the location202 associated with the user 110. As described herein, the user 110 canbe associated with a request 140 for a vehicle service. The vehiclecomputing system 104 and/or the operations computing system 106 candetermine the first vehicle route 204 so that the vehicle 104 can travelto the user 110 to provide the user 110 with the requested vehicleservice (e.g., pick up the user 110 for a transportation service,deliver an item for a delivery service, receive an item for courierservice, and/or provide another service).

At (1204), the method 1200 can include travelling along a first vehicleroute. A human operator may not be located within the vehicle 104. Thevehicle computing system 102 can provide one or more control signals tothe motion planning system 128 and/or the vehicle's control systems 116to cause the vehicle 104 to plan its motion and/or implement a motionplan to travel in accordance with first vehicle route 204 (e.g.,autonomously, without input from a human operator to the vehicle 104).

The first vehicle route 204 can bring the vehicle 104 within thevicinity 206 of the location 202 associated with the user 110. Thevicinity of the location 202 associated with the user 110 can be definedat least in part by a distance from the location 202 associated with theuser 110. In some implementations, the distance from the location 202associated with the user 110 can be based at least in part on anacceptable walking distance 302 from the location 202 associated withthe user 110, as described herein.

At (1206), the method 1200 can include determining whether a parkinglocation out of a travel way is available for the vehicle. For instance,the vehicle computing system 102 can determine whether a parkinglocation within the vicinity 206 of the location 202 associated with theuser 110 is available for the vehicle 104 (e.g., based on sensor data118, map data 120, etc.). The vehicle computing system 102 may searchfor a parking location while the vehicle 104 is in an approach operatingmode 108D, as described herein. The vehicle computing system 102 maysearch for a parking location before and/or after the vehicle 104 passesthe location 202 associated with the user 110. In the event that thevehicle computing system 102 is able to identify an available parkinglocation, the method 1200 can include sending a communication to a user110 to indicate that the vehicle 104 has arrived and the location of thevehicle 104. Once the user has arrived at the vehicle 104, the vehicle104 can provide the vehicle service to the user 110. In someimplementations, the vehicle computing system 102 can determine that aparking location that is out of the travel way 600 is unavailable forthe vehicle 104.

At (1210), the method 1200 can include obtaining traffic data associatedwith the geographic area that includes the location associated with theuser. The vehicle computing system 102 can obtain traffic data 500associated with a geographic area 200 that includes the location 202associated with the user 110. The traffic data 500 can be associatedwith the vicinity 206 of the location 202 (e.g., a block, neighborhood,etc. where the user 110 is located) and/or other portions of thegeographic area 200. For instance, the vehicle computing system 102 canobtain, via one or more sensors 112 of the vehicle 104, sensor data 118associated with the surrounding environment of the vehicle 104 that iswithin the vicinity 206 of the location 202 associated with the user110. The vehicle computing system 102 can determine a level of trafficbased at least in part on the sensor data 118, as described herein.

At (1212), the method 1200 can include determining an estimated trafficimpact. For instance, the vehicle computing system 102 can determine anestimated traffic impact 504 of the vehicle 104 on the geographic area200 based at least in part on the traffic data 500. The estimatedtraffic impact 504 can be indicative of an estimated impact of thevehicle 104 on one or more objects within a surrounding environment ofthe vehicle 104 in the event that the vehicle 104 were to stop at leastpartially in the travel way 600 (e.g., a current lane 602) within thevicinity 206 of the location 202 associated with the user 110. In someimplementations, to determine the estimated traffic impact 504, thevehicle computing system 102 can compare the level of traffic (e.g.,determined based at least in part on the sensor data, other trafficdata) to a traffic constraint 506. The traffic constraint 506 caninclude a traffic threshold indicative of a threshold level of traffic.In some implementations, the traffic constraint can be determined atleast in part from a machine-learned model, as described herein.

At (1214), the method 1200 can include obtaining location dataassociated with a user. For instance, the vehicle computing system 102can obtain location data associated with a user device 138 (e.g., mobiledevice) associated with the user 110, as described herein. The locationdata associated with the user device 138 can be indicative of one ormore locations of the user device 138 associated with the user 110 atone or more times.

At (1216), the method 1200 can include determining an estimated time ofuser arrival. For instance, the vehicle computing system 102 candetermine an estimated time of user arrival based at least in part onthe location data associated with the user device 138. The estimatedtime of user arrival can be indicative of an estimated time at which theuser 110 will arrive at the vehicle 104.

At (1218), the method 1200 can include determining one or more vehicleactions based at least in part on the estimated traffic impact and/orthe estimated time of user arrival. For instance, the vehicle computingsystem 102 can determine one or more vehicle actions based at least inpart on the estimated traffic impact 504. The vehicle computing system102 can determine the one or more vehicle actions also, oralternatively, based at least in part on the estimated time of userarrival. The one or more vehicle actions can include at least one ofstopping the vehicle 104 at least partially in a travel way 600 within avicinity 206 of the location 202 associated with the user 110 ortravelling along a second vehicle route 208 (e.g., entering into avehicle holding pattern). The second vehicle route 208 can be at leastpartially different from the first vehicle route 204. The second vehicleroute 208 can include a route that leads to the location 202 associatedwith the user 110 (or at least to a vicinity 206 of the location 202).

At (1220), the method 1200 can include causing the vehicle to performthe one or more vehicle actions. For instance, the vehicle computingsystem 102 can cause the vehicle 104 to perform the one or more vehicleactions. To do so, the vehicle computing system 102 can provide one ormore control signals to one or more systems onboard the vehicle 104 tocause the vehicle 104 to perform the vehicle action(s) (e.g., to stop inthe travel way 600, enter the vehicle holding pattern).

At (1222), the method 1200 can include providing a communication to theuser. For instance, the vehicle computing system 102 can provide acommunication to a user device 138 associated with the user 110 that isindicative of a vehicle action. By way of example, in the event that thevehicle 104 stops at least partially within a travel way 600 within thevicinity 206 of the location 202 associated with the user 110, thevehicle computing system 102 can provide, to the user device 138associated with the user 110, a communication indicating that thevehicle 104 is stopped (and/or will stop). In response to receiving thecommunication, the user device 138 can display a map user interface 1006that indicates a vehicle location of the vehicle 104 and a user route1008 to the vehicle location of the vehicle 104. In the event that thevehicle 104 travels in accordance with the second vehicle route 208, thevehicle computing system can provide a communication to the user device138 associated with the user 110 indicating that the vehicle 104 istravelling to return to the location 202 associated with the user 110.

FIGS. 13A-B depict a flow diagram of an example method 1300 ofcontrolling autonomous vehicles according to example embodiments of thepresent disclosure. One or more portion(s) of the method 1300 can beimplemented by one or more computing devices such as, for example, theone or more computing device(s) of the vehicle computing system 102and/or other systems. Each respective portion of the method 1300 (e.g.,1302-1346) can be performed by any (or any combination) of the one ormore computing devices. Moreover, one or more portion(s) of the method1300 can be implemented as an algorithm on the hardware components ofthe device(s) described herein (e.g., as in FIGS. 1 and 14), forexample, to control an autonomous vehicle. FIGS. 13A-B depicts elementsperformed in a particular order for purposes of illustration anddiscussion.

At (1302) of FIG. 13A, the vehicle 104 can enter into an approachoperating mode 108D. For example, a vehicle computing system 102 canobtain data 142 indicative of a location 202 associated with a user 110.The vehicle computing system 102 can cause the vehicle 104 to travel inaccordance with a first vehicle route 204 to arrive within a vicinity206 of the location 202 associated with the user 110. The vehicle 104can enter into the approach mode 108D, for example, when it is withinthe vicinity 206 of the location 202. The vehicle 104 can approach thelocation 202 associated with the user 110 in the approach operating mode108D. Moreover, the vehicle computing system 102 can send accurate(e.g., localized) approach data (e.g., through a network) to a softwareapplication running on a user device 138 associated with the user 110.The software application can cause the user device 138 to display a userinterface via a display device. The user interface can display a mapinterface with the user's position (e.g., based on GPS) and a preciselocation approach (e.g., of the vehicle 104), as well as a targetlocation of the vehicle 104 to meet the user 110. The user interface canalso alert the user 110 that the vehicle 104 is arriving (e.g., “yourvehicle is arriving, please prepare to board”).

In some implementations, the vehicle 104 can each include an outwardlyvisible lighting element, such as a number or array of LED lightscapable of producing rapid flash patterns. For example, the lightingelement can be located within the interior and viewable through thefront of the vehicle (e.g., windshield), and/or can be located on theexterior of the vehicle 104. When the vehicle 104 is assigned (oraccepts) a service request, the operations computing system 106 cantransmit a flash code to the vehicle 104 and the requesting user device138. As the vehicle 104 approaches the pick-up location, the vehicle 104can output the flash code using the lighting element. The requestinguser 110 can be prompted to hold up the user device 138 so that a cameraor the camera lens of the user device 138 is pointed towards the vehicle104 and can detect the flash code (e.g., the camera can be pointedtowards the vehicle 104 and the display screen of the mobile device candisplay a viewfinder or preview of the imagery detected or captured bythe camera). Upon detecting the flash code from the vehicle 104, theuser device 138 can determine whether the flash code matches the flashcode provided by the operations computing system 106 (e.g., utilizing aperception algorithm). If so, the user device 138 can display anindicator, such as a circle or a highlight for the vehicle 104, so thatthe requesting user 110 can readily identify the vehicle 104.

While in the approach operating mode 108D, the vehicle 104 can searchfor an available parking location that is out of a travel way 600 (e.g.,out of a current travel lane 602), at (1304). For example, the vehiclecomputing system 102 can search for an out-of-lane parking location whenthe vehicle 104 is within a certain distance (e.g., an acceptablewalking distance 302) from the location 202 associated with the user110. To do so, the vehicle computing system 102 can utilize thevehicle's sensor(s), as described herein.

In the event an out-of-lane parking location is found, the vehiclecomputing system 102 can cause the vehicle 104 to park (e.g.,autonomously, without user input), at (1306), and send a communicationto the user, as described herein. In some implementations, if theestimated time until the user 110 starts interaction with the vehicle104 (e.g., the estimated time until the user 110 starts boarding thevehicle 104) is low (e.g., less than a few seconds, 1, 2, 3 s, etc.) thevehicle doors can be unlocked and/or user specific vehicle settings(e.g., music, temperature, seat position, etc.) can be implemented, at(1307) (e.g., because boarding is imminent). If the user 110 arriveswithin a certain time frame (e.g., starts and/or completes boarding,item retrieval, item placement, etc. within “X” time), at (1308), thevehicle 104 can start to provide the user 110 with a vehicle service(e.g., transport the user 110 to a destination location), at (1310).However, if the user 110 does not arrive (e.g., start and/or completeboarding, item retrieval, item placement, etc.) at the vehicle 104within the time frame, the vehicle computing system 102 can againcontact the user 110, at (1312). The contact can be facilitated by thevehicle computing system 102 providing a communication to the user 110via a software application on the user device 138 (e.g., using asuitable communication protocol). If unsuccessful (e.g., the user 110 isunresponsive, does not travel to vehicle 104), the vehicle computingsystem 102 can cancel the vehicle service, at (1314), as describedherein. However, if the contact is successful, the vehicle 104 can waitanother timeframe (e.g., “Y” time) for the user 110 to arrive (e.g.,start and/or complete boarding, item retrieval, item placement, etc.),at (1316). The vehicle computing system 102 can cause the vehicle 104 toprovide the vehicle service to the user 110, in the event that the user110 arrives at the vehicle 104 (e.g., boards the vehicle 104). In theevent the user 110 does not arrive within “Y” time (e.g., start and/orcomplete boarding, item retrieval, item placement, etc.), the vehicle104 can enter into the holding pattern and travel in accordance with thesecond vehicle route 208, at (1318).

In the event that an out-of-lane parking location is not found, thevehicle computing system 102 can determine at least one of an estimatedtraffic impact 504 of the vehicle 104 on the geographic area 200 basedat least in part on traffic data 500, at (1320) or an estimated time ofuser arrival based at least in part on the location data associated withthe user device 110 as described herein, at (1322). The vehiclecomputing system 102 can determine the estimated traffic impact 504 todetermine if and how long to wait in-lane for the user 110 to arrive atthe vehicle 104. As the vehicle 104 moves towards the location 202associated with the user 110, the vehicle computing system 102 candetermine whether the vehicle 104 should stop at the location 202 basedat least in part on at least one of the estimated traffic impact 504 orthe estimated time to user arrival. For example, the vehicle computingsystem 102 can start sensing the traffic presence and object speedaround the vehicle 104 (e.g., in-lane, behind and ahead of the vehicle104). Moreover, the vehicle computing system 102 can use RF sensorsand/or Bluetooth beacons to determine a user presence, general distancebetween the user 110 and the vehicle 104, and/or changes in the distanceindicating number of seconds from user arrival.

In some implementations, in the event that the estimated traffic impact504 is low, the vehicle computing system 102 can cause the vehicle 104to stop in the travel way 600, at (1324). If the user 110 arrives (e.g.,start and/or complete boarding, item retrieval, item placement, etc.)within a certain timeframe (e.g., within “Y” time), the vehicle 104 canprovide the vehicle service to the user 110. If the user 110 does notarrive (e.g., start and/or complete boarding, item retrieval, itemplacement, etc.) within the timeframe, the vehicle 104 can enter intothe holding pattern.

In some implementations, in the event that the estimated traffic impact504 is high, the vehicle computing system 102 can determine theestimated time to user arrival. In the event that the estimated time touser arrival is low, the vehicle computing system 102 can cause thevehicle 104 to stop within the travel way 600 (e.g., in a travel lane),at (1326), despite potential traffic build-up. If the user 110 arrives(e.g., starts and/or completes boarding, item retrieval, item placement,etc.) within a certain timeframe (e.g., within “Y” time), the vehicle104 can provide the vehicle service to the user 110. If the user 110does not arrive (e.g., start and/or complete boarding, item retrieval,item placement, etc.) within the timeframe, the vehicle 104 can enterinto the holding pattern.

With reference to FIG. 13B, the vehicle computing system 102 can causethe vehicle 104 to travel past the location 202 associated with the user110, at (1328). In some implementations, this can be a portion of thesecond vehicle route 208. The vehicle computing system 102 can searchfor a parking location out of the travel way 600 after the vehicle 104passes the location 202 associated with the user 110, at (1330). Thiscan occur until the vehicle 104 travels a certain distance past thelocation 202 (e.g., until the vehicle 104 reaches the acceptable walkingdistance 302). In some implementations, even after the vehicle travelspast the location associated with the user 110, if the estimated trafficimpact 504 and/or the estimated time of user arrival is low enough thevehicle computing system 102 can cause the vehicle 104 to stop.

In the event that a parking location is found, at (1332), the vehiclecomputing system 102 can contact the user 110 via the user device 138(e.g., indicating the location of the vehicle 104 and a user routethereto). The contact can be facilitated by the vehicle computing system102 providing a communication to the user 110 via a software applicationon the user device 138 (e.g., “This is as close as I can get. Pleasecome to me”). In some implementations, if the estimated time until theuser 110 starts interaction with the vehicle 104 (e.g., the estimatedtime until the user 110 starts boarding the vehicle 104) is low (e.g.,less than a few seconds, 1, 2, 3 s, etc.) the vehicle doors can beunlocked and/or user specific vehicle settings (e.g., music,temperature, seat position, etc.) can be implemented, at (1333) (e.g.,because boarding is imminent). If the user 110 arrives (e.g., startsand/or completes boarding, item retrieval, item placement, etc.) at thevehicle 104 within a certain timeframe (e.g., with “Z” time), at (1334),the vehicle 104 can provide a vehicle service to the user 110, at(1336). If the user 110 does not arrive (e.g., start and/or completeboarding, item retrieval, item placement, etc.) within the timeframe,the vehicle computing system 102 can contact the user 110, at (1338). Ifthe contact is successful, the vehicle 104 can again wait for the user110 to arrive (e.g., start and/or complete boarding, item retrieval,item placement, etc.) to the vehicle 104. If the contact is notsuccessful, the vehicle computing system 102 can cancel the vehicleservice, at (1340).

In the event that the vehicle computing system 102 does not find aparking location after the location 202 associated with the user 110 andwithin the vicinity 206 of the location 202 (e.g., an acceptable walkingdistance 302), the vehicle computing system 102 can cause the vehicle104 to implement a vehicle holding pattern (e.g., enter a holdingpattern operating mode), at (1342). As such, the vehicle 104 can ignoreany potential parking locations and provide a communication to the userdevice 138 associated with the user 110. The user device 138 can displaya map interface depicting a location of the vehicle 104. In someimplementations, the vehicle computing system 102 can request a secondvehicle route 208 from the operations computing system 106. Theoperations computing system 106 can provide data indicative of thesecond vehicle route 208 to the vehicle computing system 102. Thevehicle computing system 102 can obtain the data indicative of thesecond vehicle route 208 and send one or more control signals to causethe vehicle 104 to travel in accordance with the second vehicle route208. At (1344), the vehicle computing system 102 can send acommunication to the user 110 indicating that the vehicle 104 istravelling to return to the location 202 associated with the user 110,as described herein. As the vehicle 104 returns back toward the location202 associated with the user 110, the vehicle 104 can enter into theapproach operating mode 108D again, at (1346). As such, the process cancontinue as shown in FIG. 13A.

The vehicle 104 can continue in the holding pattern until the vehicleservice cancellation threshold 1102 is reached. After the vehicleservice cancellation threshold 1102 is reached, the vehicle computingsystem 102 can provide a communication directly to the user device 138(e.g., via the vehicle's communication system 136) to inform the user110 that the vehicle service has been cancelled. In someimplementations, if another vehicle would be more appropriate to providethe vehicle service to the user 110 (e.g., another vehicle could arriveto the location 202 associated with the user 110 quicker than thevehicle 104), the operations computing system 106 can re-route the othervehicle to the location 202 associated with the user 110.

FIG. 14 depicts example system components of an example system 1400according to example embodiments of the present disclosure. The examplesystem 1400 can include the vehicle computing system 102, the operationscomputing system 106, and a machine learning computing system 1430 thatare communicatively coupled over one or more network(s) 1480.

The vehicle computing system 102 can include one or more computingdevice(s) 1401. The computing device(s) 1401 of the vehicle computingsystem 102 can include processor(s) 1402 and a memory 1404 (e.g.,onboard the vehicle 104). The one or more processors 1402 can be anysuitable processing device (e.g., a processor core, a microprocessor, anASIC, a FPGA, a controller, a microcontroller, etc.) and can be oneprocessor or a plurality of processors that are operatively connected.The memory 1404 can include one or more non-transitory computer-readablestorage media, such as RAM, ROM, EEPROM, EPROM, one or more memorydevices, flash memory devices, etc., and combinations thereof.

The memory 1404 can store information that can be accessed by the one ormore processors 1402. For instance, the memory 1404 (e.g., one or morenon-transitory computer-readable storage mediums, memory devices) caninclude computer-readable instructions 1406 that can be executed by theone or more processors 1402. The instructions 1406 can be softwarewritten in any suitable programming language or can be implemented inhardware. Additionally, or alternatively, the instructions 1406 can beexecuted in logically and/or virtually separate threads on processor(s)1402.

For example, the memory 1404 can store instructions 1406 that whenexecuted by the one or more processors 1402 cause the one or moreprocessors 1402 (the computing system 102) to perform operations such asany of the operations and functions of the vehicle computing system 102,the vehicle 104, or for which the vehicle computing system 102 and/orthe vehicle 104 are configured, as described herein, the operations fordetermining autonomous boarding times and/or other time estimates (e.g.,one or more portions of method 700) the operations for controllingautonomous vehicles (e.g., one or more portions of methods 1200 and/or1300), and/or any other functions for the vehicle 104, as describedherein.

The memory 1404 can store data 1408 that can be obtained, received,accessed, written, manipulated, created, and/or stored. The data 1408can include, for instance, traffic data, location data, historic data,map data, sensor data, state data, prediction data, motion planningdata, data associated with operating modes, data associated withestimated times, and/or other data/information described herein. In someimplementations, the computing device(s) 1401 can obtain data from oneor more memory device(s) that are remote from the vehicle 104.

The computing device(s) 1401 can also include a communication interface1409 used to communicate with one or more other system(s) on-board thevehicle 104 and/or a remote computing device that is remote from thevehicle 104 (e.g., the other systems of FIG. 1400, a user deviceassociated with a user, etc.). The communication interface 1409 caninclude any circuits, components, software, etc. for communicating viaone or more networks (e.g., 1480). In some implementations, thecommunication interface 1409 can include for example, one or more of acommunications controller, receiver, transceiver, transmitter, port,conductors, software and/or hardware for communicating data/information.

The operations computing system 106 can perform the operations andfunctions for managing autonomous vehicles, as described herein. Theoperations computing system 106 can be located remotely from the vehicle104. For example, the operations computing system 106 can operateoffline, off-board, etc. The operations computing system 106 can includeone or more distinct physical computing devices.

The operations computing system 106 can include one or more computingdevices 1420. The one or more computing devices 1420 can include one ormore processors 1422 and a memory 1424. The one or more processors 1422can be any suitable processing device (e.g., a processor core, amicroprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.)and can be one processor or a plurality of processors that areoperatively connected. The memory 1424 can include one or morenon-transitory computer-readable storage media, such as RAM, ROM,EEPROM, EPROM, one or more memory devices, flash memory devices, etc.,and combinations thereof.

The memory 1424 can store information that can be accessed by the one ormore processors 1422. For instance, the memory 1424 (e.g., one or morenon-transitory computer-readable storage mediums, memory devices) canstore data 1426 that can be obtained, received, accessed, written,manipulated, created, and/or stored. The data 1426 can include, forinstance, service request data, vehicle data, vehicle servicecancellation thresholds, and/or other data or information describedherein. In some implementations, the operations computing system 106 canobtain data from one or more memory device(s) that are remote from theoperations computing system 106.

The memory 1424 can also store computer-readable instructions 1428 thatcan be executed by the one or more processors 1422. The instructions1428 can be software written in any suitable programming language or canbe implemented in hardware. Additionally, or alternatively, theinstructions 1428 can be executed in logically and/or virtually separatethreads on processor(s) 1422.

For example, the memory 1424 can store instructions 1428 that whenexecuted by the one or more processors 1422 cause the one or moreprocessors 1422 to perform any of the operations and/or functionsdescribed herein, including, for example, any of the operations andfunctions of the operations computing system 106, the computingdevice(s) 1420, and any of the operations and functions for which theoperations computing system 106 and/or the computing device(s) 1420 areconfigured, as described herein, as well as one or more portions ofmethods 1200 and/or 1300.

The computing device(s) 1420 can also include a communication interface1429 used to communicate with one or more other system(s). Thecommunication interface 1429 can include any circuits, components,software, etc. for communicating via one or more networks (e.g., 1480).In some implementations, the communication interface 1429 can includefor example, one or more of a communications controller, receiver,transceiver, transmitter, port, conductors, software and/or hardware forcommunicating data/information.

According to an aspect of the present disclosure, the vehicle computingsystem 102 and/or the operations computing system 106 can store orinclude one or more machine-learned models 1440. As examples, themachine-learned models 1440 can be or can otherwise include variousmachine-learned models such as, for example, neural networks (e.g., deepneural networks), support vector machines, decision trees, ensemblemodels, k-nearest neighbors models, Bayesian networks, or other types ofmodels including linear models and/or non-linear models. Example neuralnetworks include feed-forward neural networks, recurrent neural networks(e.g., long short-term memory recurrent neural networks), or other formsof neural networks.

In some implementations, the vehicle computing system 102 and/or theoperations computing system 106 can receive the one or moremachine-learned models 1440 from the machine learning computing system1430 over the network(s) 1480 and can store the one or moremachine-learned models 1440 in the memory of the respective system. Thevehicle computing system 102 and/or the operations computing system 106can use or otherwise implement the one or more machine-learned models1440 (e.g., by processor(s) 1402, 1422). In particular, the vehiclecomputing system 102 and/or the operations computing system 106 canimplement the machine learned model(s) 1440 to determine an acceptablewalking distance, traffic constraint, estimated traffic impact,estimated time of user arrival (e.g., estimated boarding time, estimatedboarding during, estimated boarding completion, etc.), second vehicleroute (e.g., vehicle holding pattern), vehicle service cancellationthreshold, etc., as described herein.

The machine learning computing system 1430 can include one or moreprocessors 1432 and a memory 1434. The one or more processors 1432 canbe any suitable processing device (e.g., a processor core, amicroprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.)and can be one processor or a plurality of processors that areoperatively connected. The memory 1434 can include one or morenon-transitory computer-readable storage media, such as RAM, ROM,EEPROM, EPROM, one or more memory devices, flash memory devices, etc.,and combinations thereof.

The memory 1434 can store information that can be accessed by the one ormore processors 1432. For instance, the memory 1434 (e.g., one or morenon-transitory computer-readable storage mediums, memory devices) canstore data 1436 that can be obtained, received, accessed, written,manipulated, created, and/or stored. In some implementations, themachine learning computing system 1430 can obtain data from one or morememory devices that are remote from the system 1430.

The memory 1434 can also store computer-readable instructions 1438 thatcan be executed by the one or more processors 1432. The instructions1438 can be software written in any suitable programming language or canbe implemented in hardware. Additionally, or alternatively, theinstructions 1438 can be executed in logically and/or virtually separatethreads on processor(s) 1432. The memory 1434 can store the instructions1438 that when executed by the one or more processors 1432 cause the oneor more processors 1432 to perform operations.

In some implementations, the machine learning computing system 1430 caninclude one or more server computing devices. If the machine learningcomputing system 1430 includes multiple server computing devices, suchserver computing devices can operate according to various computingarchitectures, including, for example, sequential computingarchitectures, parallel computing architectures, or some combinationthereof.

In addition or alternatively to the model(s) 1440 at the vehiclecomputing system 102 and/or the operations computing system 106, themachine learning computing system 1430 can include one or moremachine-learned models 1450. As examples, the machine-learned models1450 can be or can otherwise include various machine-learned models suchas, for example, neural networks (e.g., deep neural networks), supportvector machines, decision trees, ensemble models, k-nearest neighborsmodels, Bayesian networks, or other types of models including linearmodels and/or non-linear models. Example neural networks includefeed-forward neural networks, recurrent neural networks (e.g., longshort-term memory recurrent neural networks, or other forms of neuralnetworks. The machine-learned models 1450 can be similar to and/or thesame as the machine-learned models 1440.

As an example, the machine learning computing system 1430 cancommunicate with the vehicle computing system 102 and/or the operationscomputing system 106 according to a client-server relationship. Forexample, the machine learning computing system 1430 can implement themachine-learned models 1450 to provide a web service to the vehiclecomputing system 102 and/or the operations computing system 106. Forexample, the web service can provide machine-learned models to an entityassociated with an autonomous vehicle; such that the entity canimplement the machine-learned model (e.g., to determine estimatedtraffic impacts, vehicle service request cancellation, etc.). Thus,machine-learned models 1450 can be located and used at the vehiclecomputing system 102 and/or the operations computing system 106 and/ormachine-learned models 1450 can be located and used at the machinelearning computing system 1430.

In some implementations, the machine learning computing system 1430, thevehicle computing system 102, and/or the operations computing system 106can train the machine-learned models 1440 and/or 1450 through use of amodel trainer 1460. The model trainer 1460 can train the machine-learnedmodels 1440 and/or 1450 using one or more training or learningalgorithms. One example training technique is backwards propagation oferrors. In some implementations, the model trainer 1460 can performsupervised training techniques using a set of labeled training data. Inother implementations, the model trainer 1460 can perform unsupervisedtraining techniques using a set of unlabeled training data. The modeltrainer 1460 can perform a number of generalization techniques toimprove the generalization capability of the models being trained.Generalization techniques include weight decays, dropouts, or othertechniques.

In particular, the model trainer 1460 can train a machine-learned model1440 and/or 1450 based on a set of training data 1462. The training data1462 can include, for example, a number of sets of data from previousevents (e.g., acceptable walking distance data, historic traffic data,user arrival data, trip cancellation data, user feedback data, otherdata described herein). In some implementations, the training data 1462can be taken from the same geographic area (e.g., city, state, and/orcountry) in which an autonomous vehicle utilizing that model 1440/1450is designed to operate. In this way, the models 1450/1450 can be trainedto determine outputs (e.g., estimated traffic impact, acceptable walkingdistances) in a manner that is tailored to the customs of a particularlocation (e.g., waiting for a user longer, decreasing an acceptablewalking distance, etc.). The model trainer 1460 can be implemented inhardware, firmware, and/or software controlling one or more processors.

The network(s) 1480 can be any type of network or combination ofnetworks that allows for communication between devices. In someembodiments, the network(s) 1480 can include one or more of a local areanetwork, wide area network, the Internet, secure network, cellularnetwork, mesh network, peer-to-peer communication link and/or somecombination thereof and can include any number of wired or wirelesslinks. Communication over the network(s) 1480 can be accomplished, forinstance, via a network interface using any type of protocol, protectionscheme, encoding, format, packaging, etc.

FIG. 14 illustrates one example system 1400 that can be used toimplement the present disclosure. Other computing systems can be used aswell. For example, in some implementations, the vehicle computing system102 and/or the operations computing system 106 can include the modeltrainer 1460 and the training dataset 1462. In such implementations, themachine-learned models 1440 can be both trained and used locally at thevehicle computing system 102 and/or the operations computing system 106.As another example, in some implementations, the vehicle computingsystem 102 and/or the operations computing system 104 may not beconnected to other computing systems.

Computing tasks discussed herein as being performed at computingdevice(s) remote from the vehicle can instead be performed at thevehicle (e.g., via the vehicle computing system), or vice versa. Suchconfigurations can be implemented without deviating from the scope ofthe present disclosure. The use of computer-based systems allows for agreat variety of possible configurations, combinations, and divisions oftasks and functionality between and among components.Computer-implemented operations can be performed on a single componentor across multiple components. Computer-implemented tasks and/oroperations can be performed sequentially or in parallel. Data andinstructions can be stored in a single memory device or across multiplememory devices.

While the present subject matter has been described in detail withrespect to specific example embodiments and methods thereof, it will beappreciated that those skilled in the art, upon attaining anunderstanding of the foregoing can readily produce alterations to,variations of, and equivalents to such embodiments. Accordingly, thescope of the present disclosure is by way of example rather than by wayof limitation, and the subject disclosure does not preclude inclusion ofsuch modifications, variations and/or additions to the present subjectmatter as would be readily apparent to one of ordinary skill in the art.

What is claimed is:
 1. A computer-implemented method of controllingautonomous vehicles, comprising: obtaining, by a vehicle computingsystem that comprises one or more computing devices, data indicative ofa location associated with a user to which an autonomous vehicle is totravel, wherein the autonomous vehicle is to travel along a firstvehicle route that leads to the location associated with the user;obtaining, by the computing system via one or more sensors of theautonomous vehicle, sensor data associated with a surroundingenvironment of the autonomous vehicle that is within a vicinity of thelocation associated with the user; determining, by the computing system,a level of traffic associated with a geographic area that includes thelocation associated with the user based at least in part on the sensordata; determining, by the vehicle computing system, an estimated trafficimpact of the autonomous vehicle on the geographic area based at leastin part on the level of traffic that is based at least in part on thesensor data, wherein the estimated traffic impact is indicative of anestimated impact of the autonomous vehicle on one or more objects withinthe surrounding environment of the autonomous vehicle in the event thatthe autonomous vehicle were to stop at least partially in a travel waywithin the vicinity of the location associated with the user;determining, by the vehicle computing system, one or more vehicleactions based at least in part on the estimated traffic impact; andcausing, by the vehicle computing system, the autonomous vehicle toperform the one or more vehicle actions, wherein the one or more vehicleactions comprise at least one of stopping the autonomous vehicle atleast partially in the travel way within the vicinity of the locationassociated with the user or travelling along a second vehicle route. 2.The computer-implemented method of claim 1, further comprising:obtaining, by the vehicle computing system, location data associatedwith a user device associated with the user; determining, by the vehiclecomputing system, an estimated time of user arrival based at least inpart on the location data associated with the user device; anddetermining, by the vehicle computing system, the one or more vehicleactions also based at least in part on the estimated time of userarrival.
 3. The computer-implemented method of claim 1, wherein thevicinity of the location associated with the user is defined at least inpart by a distance from the location associated with the user, andwherein the distance from the location associated with the user is basedat least in part on an acceptable walking distance from the locationassociated with the user.
 4. The computer-implemented method of claim 1,wherein determining, by the vehicle computing system, the estimatedtraffic impact comprises: comparing, by the vehicle computing system,the level of traffic to a traffic constraint.
 5. Thecomputer-implemented method of claim 4, wherein the traffic constraintcomprises a traffic threshold indicative of a threshold level oftraffic.
 6. The computer-implemented method of claim 4, wherein thetraffic constraint is determined at least in part on a machine-learnedmodel.
 7. The computer-implemented method of claim 1, wherein the secondvehicle route is at least partially different from the first vehicleroute and wherein the second vehicle route includes a route that leadsto the location associated with the user.
 8. The computer-implementedmethod of claim 1, wherein the autonomous vehicle stops at leastpartially in the travel way within the vicinity of the locationassociated with the user, the method further comprising: providing, bythe vehicle computing system to a user device associated with the user,a communication indicating that the autonomous vehicle is stopped, andwherein in response to receiving the communication the user devicedisplays a map user interface that indicates a vehicle location of theautonomous vehicle and a user route to the vehicle location of theautonomous vehicle.
 9. The computer-implemented method of claim 1,further comprising: determining, by the vehicle computing system, that aparking location that is out of the travel way is unavailable for theautonomous vehicle.
 10. A vehicle computing system onboard an autonomousvehicle, comprising: one or more processors; and one or more memorydevices, the one or more memory devices storing instructions that whenexecuted by the one or more processors cause the computing system toperform operations, the operations comprising: obtaining data indicativeof a location associated with a user, wherein the user is associatedwith a request for a vehicle service provided by the autonomous vehicle,and wherein the autonomous vehicle is to travel along a first vehicleroute to arrive within a vicinity of the location associated with theuser; obtaining, via one or more sensors of the autonomous vehicle,sensor data associated with a surrounding environment of the autonomousvehicle that is within the vicinity of the location associated with theuser; determining a level of traffic associated with a geographic areathat includes the location associated with the user based at least inpart on the sensor data; obtaining location data associated with a userdevice associated with the user; determining an estimated traffic impactof the autonomous vehicle on the geographic area based at least in parton the level of traffic that is based at least in part on the sensordata and an estimated time of user arrival based at least in part on thelocation data associated with the user device, wherein the estimatedtraffic impact is indicative of an estimated impact of the autonomousvehicle on one or more objects within the surrounding environment of theautonomous vehicle in the event that the autonomous vehicle were to stopat least partially in a travel way within the vicinity of the locationassociated with the user; determining one or more vehicle actions basedat least in part on at least one of the estimated traffic impact or theestimated time of user arrival; and causing the autonomous vehicle toperform the one or more vehicle actions, wherein the one or more vehicleactions comprise at least one of stopping the autonomous vehicle atleast partially in the travel way within the vicinity of the locationassociated with the user or travelling along a second vehicle route. 11.The computing system of claim 10, wherein determining one or morevehicle actions based at least in part on at least one of the estimatedtraffic impact or the estimated time of user arrival comprises: applyinga first weighting factor to the estimated traffic impact and a secondweighting factor to the estimated time of user arrival.
 12. Thecomputing system of claim 10, wherein the autonomous vehicle stops atleast partially in the travel way within the vicinity of the locationassociated with the user, and wherein the operations further comprise:determining at least one of an updated estimated traffic impact or anupdated estimated time of user arrival; and causing the autonomousvehicle to travel along the second vehicle route based at least in parton at least one of the updated estimated traffic impact or the updatedestimated time of user arrival.
 13. The computing system of claim 10,wherein the autonomous vehicle travels along the second vehicle route,and wherein the operations further comprise: providing a communicationto the user device associated with the user, wherein the communicationindicates that the autonomous vehicle is travelling to return to thelocation associated with the user.
 14. The computing system of claim 10,wherein the user does not arrive at the autonomous vehicle, and whereinthe operations further comprise: providing, to one or more computingdevices that are remote from the computing system, data indicating thatthe request for the vehicle service provided by the autonomous vehicleis cancelled.
 15. An autonomous vehicle comprising: one or more sensors;a communication system; one or more processors; and one or more memorydevices, the one or more memory devices storing instructions that whenexecuted by the one or more processors cause the autonomous vehicle toperform operations, the operations comprising: obtaining data indicativeof a location associated with a user, wherein the user is associatedwith a request for a vehicle service provided by the autonomous vehicle;controlling the autonomous vehicle to travel along a first vehicle routeto arrive within a vicinity of the location associated with the user;obtaining, via the one or more sensors, sensor data associated with asurrounding environment of the autonomous vehicle that is within thevicinity of the location associated with the user; determining a levelof traffic associated with a geographic area that includes the locationassociated with the user based at least in part on the sensor data;obtaining, via the communication system, location data associated with auser device associated with the user, wherein the location dataassociated with the user device is indicative of one or more locationsof the user device associated with the user at one or more times;determining an estimated traffic impact of the autonomous vehicle on thegeographic area based at least in part on the traffic data, wherein theestimated traffic impact is indicative of an estimated impact of theautonomous vehicle on one or more objects within the surroundingenvironment of the autonomous vehicle in the event that the autonomousvehicle were to stop at least partially in a travel way within thevicinity of the location associated with the user; determining anestimated time of user arrival based at least in part on the locationdata associated with the user device; determining one or more vehicleactions based at least in part on at least one of the estimated trafficimpact or the estimated time of user arrival; and causing the autonomousvehicle to perform the one or more vehicle actions, wherein the one ormore vehicle actions comprise at least one of stopping the autonomousvehicle within the vicinity of the location associated with the user ortravelling along a second vehicle route.
 16. The autonomous vehicle ofclaim 15, wherein causing the autonomous vehicle to stop within thevicinity of the location associated with the user comprises: providingone or more control signals to a control system of the autonomousvehicle to cause the autonomous vehicle to decelerate to a stoppedposition that is at least partially in a travel way within the vicinityof the location associated with the user.
 17. The autonomous vehicle ofclaim 15, wherein causing the autonomous vehicle to travel along thesecond vehicle route comprises: providing one or more control signals tocause the autonomous vehicle to implement a motion plan that causes theautonomous vehicle to travel in accordance with the second vehicleroute.
 18. The autonomous vehicle of claim 15, wherein obtaininglocation data associated with the user device comprises: obtaining thelocation data associated with user device using multiple input, multipleoutput communication between the autonomous vehicle and the user deviceassociated with the user.